The main object of study in this chapter is (temporally homogeneous) Markov chains on a countable state space S. That is, a sequence of r.v.’s Xn, n ≥ 0, with
where = σ(X0, . . ., Xn), p(i, j) ≥ 0 and Σj p(i, j) = 1. The theory focuses on the asymptotic behavior of pn(i, j) ≡ P(Xn = j|X0 = i). The basic results are that
and under a mild assumption called aperiodicity:
In nice situations, that is, Xn is irreducible and positive ...