To describe the evolution of a system, we must prescribe how the future depends on the present or the past. Two major examples of such descriptions are differential equations and recurrent sequences. When a piece of randomness is added, it leads to stochastic differential equations (which are beyond the scope of this book) and stochastic recurrent sequences (SRS), which we already studied in Chapter 2. Among SRS, Markov chains are the most salient category. Behind a seemingly simple description lies a mathematical tool which is quite efficient for applications and rich of many properties.

Remember that it is recommended to read section A.1.1.

Consider a sequence of random variables *X* = (*X _{n}*,

Trajectories of *X* are elements of *E*^{N}, that is to say sequences of elements of *E*. The shift (see section A) is then defined by

This shift is the non-bijective restriction to *E*^{N} of the bijective flow defined on *E*^{Z}in section 2.1. As with the flow, we need to define the *n*th iteration of *θ*, denoted as *θ*,^{n}and defined by

From now on, we identify *θ* and *θ*^{1}.

DEFINITION 3.1.– *The sequence X is a Markov chain when for any n* ≤ *m, theσ-field ...*

Start Free Trial

No credit card required