Markov Chains in Random Environments: The Case of Markovian Environments

A formulation of a Markov chain in a random environment is given, generalizing special cases such as branching processes, queues, birth and death chains and random walks in random environments. It is assumed that the environmental process is Markovian, each environment corresponding to a particular law of evolution on a countable state space $\mathscr{X}$. It is then shown that there is a natural three way classification of states of $\mathscr{X}$. One of the three types of states is irregular in nature, and conditions are found under which no such states exist.