A two-point Markov chain boundary-value problem

Publisher Website
Google Scholar
Abstract
The two-point Markov chain boundary-value problem discussed in this paper is a finite-time version of the quasi-stationary behaviour of Markov chains. Specifically, for a Markov chain {Xt:t= 0, 1, ·· ·}, given the time interval (0,n), the interest is in describing the chain at some intermediate time point rconditional on knowing boththe behaviour of the chain at the initial time point 0 and that over the interval (0,n) it has avoided some subsetBof the state space. The paper considers both ‘real time' estimates forr=n(i.e. the chain has avoidedBsince 0), anda posterioriestimates forr<nwith at least partial knowledge of the behaviour ofXn.Algorithms to evaluate the distribution ofXrcan be as small asO(n3) (and, for practical purposes, evenO(n2logn)). The estimates may be stochastically ordered, and the process (and hence, the estimates) may be spatially homogeneous in a certain sense. Maximum likelihood estimates of the sample path are furnished, but by example we note that these ML paths may differ markedly from the path consisting of the expected or average states. The scope for two-point boundary-value problems to have solutions in a Markovian setting is noted.Several examples are given, together with a discussion and examples of the analogous problem in continuous time. These examples include the basicM/G/kqueue and variants that include a finite waiting room, reneging, balking, and Bernoulli feedback, a pure birth process and the Yule process. The queueing examples include Larson's (1990) ‘queue inference engine'.