O'Reilly logo

Probability, Fourth Edition by Rick Durrett

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

6Markov Chains

The main object of study in this chapter is (temporally homogeneous) Markov chains on a countable state space S. That is, a sequence of r.v.’s Xn, n ≥ 0, with

image

where Images = σ(X0, . . ., Xn), p(i, j) ≥ 0 and Σj p(i, j) = 1. The theory focuses on the asymptotic behavior of pn(i, j) ≡ P(Xn = j|X0 = i). The basic results are that

image

and under a mild assumption called aperiodicity:

In nice situations, that is, Xn is irreducible and positive ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required