The optimality equation in average cost denumerable state semi-Markov decision problems, recurrency conditions and algorithms
- 1 June 1978
- journal article
- Published by Cambridge University Press (CUP) in Journal of Applied Probability
- Vol. 15 (2) , 356-373
- https://doi.org/10.2307/3213407
Abstract
This paper is concerned with the optimality equation for the average costs in a denumerable state semi-Markov decision model. It will be shown that under each of a number of recurrency conditions on the transition probability matrices associated with the stationary policies, the optimality equation has a bounded solution. This solution indeed yields a stationary policy which is optimal for a strong version of the average cost optimality criterion. Besides the existence of a bounded solution to the optimality equation, we will show that both the value-iteration method and the policy-iteration method can be used to determine such a solution. For the latter method we will prove that the average costs and the relative cost functions of the policies generated converge to a solution of the optimality equation.Keywords
This publication has 15 references indexed in Scilit:
- Contraction mappings underlying undiscounted Markov decision problemsJournal of Mathematical Analysis and Applications, 1978
- Exponential convergence of products of stochastic matricesJournal of Mathematical Analysis and Applications, 1977
- Sensitive Optimality Criteria in Countable State Dynamic ProgrammingMathematics of Operations Research, 1977
- Conditions for the Equivalence of Optimality Criteria in Dynamic ProgrammingThe Annals of Statistics, 1976
- Markov decision chains with unbounded costs and applications to the control of queuesAdvances in Applied Probability, 1976
- The asymptotic behaviour of the minimal total expected cost for the denumerable state Markov decision modelJournal of Applied Probability, 1975
- Iterative solution of the functional equations of undiscounted Markov renewal programmingJournal of Mathematical Analysis and Applications, 1971
- A Solution to a Countable System of Equations Arising in Markovian Decision ProcessesThe Annals of Mathematical Statistics, 1967
- Denumerable State Markovian Decision Processes-Average Cost CriterionThe Annals of Mathematical Statistics, 1966
- Weak ergodicity in non-homogeneous Markov chainsMathematical Proceedings of the Cambridge Philosophical Society, 1958