The average-optimal adaptive control of a Markov renewal model in presence of an unknown parameter
- 1 January 1982
- journal article
- research article
- Published by Taylor & Francis in Mathematische Operationsforschung und Statistik. Series Optimization
- Vol. 13 (4) , 567-591
- https://doi.org/10.1080/02331938208842822
Abstract
We consider a Markov renewal decision model with countable state space and an unbounded reward function. The transition probability depends on an unknown parameter θ. We give weak bounding and recurrence conditions under which the adaptation of the control according to the outcome of an estimator for θ yields an optimal procedure uniformly in θ with respect to the expected average reward criterion In a detailed example these results are applied to a controllable M/G/1 queueing model with unknown arrival rate and unknown service time distribution.Keywords
This publication has 15 references indexed in Scilit:
- Optimal Control of Semi-Markov Chains under Uncertainty with Applications to Queueing ModelsPublished by Springer Nature ,1980
- On monotone optimal policies in a queueing model ofM/G/1 type with controllable service time distributionAdvances in Applied Probability, 1979
- Denumerable state semi-Markov decision processes with unbounded costs, average cost criterionStochastic Processes and their Applications, 1979
- Markov Strategies in Dynamic ProgrammingMathematics of Operations Research, 1978
- Stationary Markovian Decision Problems and Perturbation Theory of Quasi-Compact Linear OperatorsMathematics of Operations Research, 1977
- Regenerative Markov decision modelsPublished by Springer Nature ,1976
- On dynamic programming: Compactness of the space of policiesStochastic Processes and their Applications, 1975
- Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimalProbability Theory and Related Fields, 1975
- An adaptive automaton controller for discrete-time markov processesAutomatica, 1969
- A Note on Memoryless Rules for Controlling Sequential Control ProcessesThe Annals of Mathematical Statistics, 1966