An algorithmic approach for a special class of Markov chains