Aggregation and optimal control of nearly completely decomposable Markov chains
- 7 January 2003
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1277-1282 vol.2
- https://doi.org/10.1109/cdc.1989.70343
Abstract
A study is made of the aggregation of nearly completely decomposable Markov chains. Three results are obtained. First, a similarity transformation that transforms the system into a singularly perturbed form is given. Second, an aggregation method for computing the exact steady-state probability distribution, as well as its O( in /sup k/) approximations, is derived. Third, a policy improvement algorithm based on the above aggregation method is developed for computing the optimal control that minimizes the average cost per stage over infinite horizon.Keywords
This publication has 6 references indexed in Scilit:
- Iterative aggregation/disaggregation techniques for nearly uncoupled markov chainsJournal of the ACM, 1985
- Aggregation with an error of O (ε 2 )Journal of the ACM, 1985
- Iterative Methods for Computing Stationary Distributions of Nearly Completely Decomposable Markov ChainsSIAM Journal on Algebraic Discrete Methods, 1984
- A singular perturbation approach to modeling and control of Markov chainsIEEE Transactions on Automatic Control, 1981
- Optimal control of markov chains admitting strong and weak interactionsAutomatica, 1981
- Control of weakly-coupled Markov chainsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1980