Discretization and Weak Convergence in Markov Decision Drift Processes

Abstract
In this paper we deal with continuous time Markov decision drift processes (CTMDP), which permit both controls affecting jump rates of the process and impulsive controls causing immediate transitions. Between two successive jump epochs the state of the process evolves according to a deterministic drift function. Given a CTMDP we construct a sequence of discrete time Markov decision drift processes (DTMDP) with decreasing distance between the successive decision epochs. Sufficient conditions are provided under which the law of the CTMDP controlled by a fixed policy is the limit (in the sense of weak convergence of probability measures) of the laws of the approximating DTMDF's controlled by fixed discrete time policies. The conditions concern both the parameters of the CTMDP and the relation between the discrete time and continuous time policies. An application to a maintenance replacement model is given.

This publication has 0 references indexed in Scilit: