Markov Processes: Linguistics and Zipf's Law

It is shown that a 2-parameter random Markov process constructed with $N$ states and biased random transitions gives rise to a stationary distribution where the probabilities of occurrence of the states, $P\left(k\right),k=1,\dots ,N$, exhibit the following three universal behaviors which characterize biological sequences and texts in natural languages: (a) the rank-ordered frequencies of occurrence of words are given by Zipf's law $P\left(k\right)\propto 1/k^{\rho }$, where $\rho \left(k\right)$ is slowly increasing for small $k$; (b) the frequencies of occurrence of letters are given by $P\left(k\right)=A-D\ln \left(k\right)$; and (c) long-range correlations are observed over long but finite intervals, as a result of the quasiergodicity of the Markov process.