Relation between continuous and discrete time markovian decision problems
- 1 September 1977
- journal article
- Published by Wiley in Naval Research Logistics Quarterly
- Vol. 24 (3) , 431-439
- https://doi.org/10.1002/nav.3800240306
Abstract
An associated discrete (continuous) time Markovian decision problem for any given continuous (discrete) time Markovian decision model is formulated. A relationship between the continuous and associated discrete time discounted returns is obtained. This result is used to show the existence of a deterministic stationary policy that optimizes both discounted return functions. It is also proved that the same policy optimizes the average expected return for both continuous and the associated discrete time processes. The results obtained in this paper can be used to solve the continuous time Markovian decision problem by using the discrete time algorithms or vice versa.Keywords
This publication has 4 references indexed in Scilit:
- Continuous time Markovian decision processes average return criterionJournal of Mathematical Analysis and Applications, 1975
- Nondiscounted Continuous Time Markovian Decision Process with Countable State SpaceSIAM Journal on Control, 1972
- Continuously Discounted Markov Decision Model with Countable State and Action SpaceThe Annals of Mathematical Statistics, 1971
- Denumerable State Markovian Decision Processes-Average Cost CriterionThe Annals of Mathematical Statistics, 1966