Countable-State, Continuous-Time Dynamic Programming with Structure

Publisher Website
Google Scholar
Abstract
We consider the problem P of maximizing the expected discounted reward earned in a continuous-time Markov decision process with countable state and finite action space. (The reward rate is merely bounded by a polynomial.) By examining a sequence 〈pN〉 of approximating problems, each of which is a semi-Markov decision process with exponential transition rate ΛN, ΛN ↗ ∞, we are able to show that P is obtained as the limit of the PN. The value in representing P as the limit of PN is that structural properties present in each PN persist, in both the finite and the infinite horizon problem. Three queuing optimization models illustrating the method are given.

This publication has 0 references indexed in Scilit: