Some Studies of Denumerable Markov Chains and a Queueing Model

V.S.S. Kumar (1987). Some Studies of Denumerable Markov Chains and a Queueing Model. Journal of Statistical Research, Vol. 21, No. 1-2, pp. ??-??.

Abstract

Let $\{X_n, n\geq0\}$ be an irreducible time-homogeneous Marov chain with transition probability matrix $P=\{p_{ij}\}$ . The quantity $E(X_{n+1}-X_n\mid X_n=i)$ is of interest because it gives intuitively appealing and easily computed conditions for positive recurrence. The condition $\lim sup_{i\rightarrow \infty}E(X_{n+1}-X_n\mid X_n=i)<0$ was given by Pakes and the main result of this paper is a sharpening of this criterion for periodic Markov chains with transition probabilities satisfying $\lim_{i\rightarrow \infty}p_{ij}=0$ for each $j$ . This facilitated by considering a Markov chain for which the matrix $p$ can be written as the product of a finite number of stochastic matrices and obtaining an inequality satisfied by $\lim sup_{i\rightarrow \infty}E(X_{n+1}-X_n\mid X_n=i)$ . This inequality can also be used directly to get conditions for positive recurrence of such a Markov chain and this is illustrated in the last section by considering a single server queuing problem with rotating servers.