Mathematical programming and the control of Markov chains†

1 May 1971

journal article
research article
Published by Taylor & Francis in International Journal of Control

Vol. 13 (5) , 801-820
https://doi.org/10.1080/00207177108931985

Abstract

Linear programming versions of some control problems on Markov chains are derived, and are studied under conditions which occur in typical problems which arise by discretizing continuous time and state systems or in discrete state systems. Control interpretations of the dual variables and simplex multipliers are given. The formulations allows the treatment of ‘state space’ like constraints which cannot be handled conveniently with dynamic programming. The relation between dynamic programming on Markov chains and the deterministic discrete maximum principle is explored, and some insight is obtained into the problem of singular stochastic controls (with respect to a stochastic maximum principle).

Keywords

This publication has 3 references indexed in Scilit:

Numerical methods for the solution of the degenerate nonlinear elliptic equations arising in optimal stochastic control theory
IEEE Transactions on Automatic Control, 1968
On the stochastic maximum principle with “average” constraints
Journal of Mathematical Analysis and Applications, 1965
Linear Programming in a Markov Chain
Operations Research, 1962