Estimation and control in Markov chains

1 March 1974

journal article
research article
Published by Cambridge University Press (CUP) in Advances in Applied Probability

Vol. 6 (01) , 40-60
https://doi.org/10.1017/s0001867800039707

Abstract

We consider a finite controlled Markov chain, the description of which depends on an unknown parameter a, and investigate the following control policy. To each a an optimal stationary control is associated. a is estimated recurrently from the trajectory by the minimum contrast method, and the optimal stationary control corresponding to the estimate is used. We present asymptotic properties of the estimate and of the criterion function. They follow from the law of large numbers and from the central limit theorem for controlled Markov chains derived with the aid of martingales.

Keywords

This publication has 5 references indexed in Scilit:

On the adaptive control of finite state Markov processes
Probability Theory and Related Fields, 1973
Note on minimum contrast estimates forMarkov processes
Metrika, 1972
Martingale Central Limit Theorems
The Annals of Mathematical Statistics, 1971
Martingale convergence to infinitely divisible laws with finite variances
Transactions of the American Mathematical Society, 1971
On the measurability and consistency of minimum contrast estimates
Metrika, 1969