O'Reilly logo

A First Course in Probability and Markov Chains by Laura Poggiolini, Giuseppe Modica

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

6

An introduction to continuous time Markov chains

In this chapter, we give a very short introduction to continuous time Markov chains. In Section 6.1 we discuss the Poisson process while in Section 6.2 we consider homogeneous continuous time Markov chains with finite state-space and right-continuous trajectories. The convergence to equilibrium of the transition probabilities matrices and the description of the holding times are discussed.

6.1   Poisson process

We go back to Example 3.34 by giving an explicit example: the emission of clicks of a Geiger counter. In this section we follow closely the presentation in [5].

Evidence shows the following features of the distribution of the number of clicks produced by the counter:

(i) The number of clicks produced in pairwise disjoint time intervals are independent.

(ii) The clicks are uniformly distributed with respect to time, i.e. the number of clicks produced in the intervals [a + c, b + c] a, b, c ≥ 0 does not depend on c.

(iii) In each time interval, the average number of clicks is finite,

(iv) At each moment, the Geiger counter produces at most one click.

When defining a model, the previous features can be formalized as follows. Let images be the family of time intervals

Images

Consider the Geiger counter as a probability space (Ω, ε, ) and, ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required