O'Reilly logo

Stochastic Modeling and Analysis of Telecoms Networks by Pascal Moyal, Laurent Decreusefond

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 7Markov Process

 

 

 

The Markovian modeling of a dynamic system often leads to a Markov chain, for which the sojourn time in each state becomes random. In many cases, this description is insufficient to establish interesting mathematical properties. In that purpose, we introduce the formalism of Markov jump processes, with their semi-groups and infinitesimal generators.

To go one step further and, in particular, to prove several crucial results such as PASTA or its avatars, we need to see a Markov process as the solution of a martingale problem. In this chapter, we review these different characterizations, and show that they are in fact equivalent.

Throughout this chapter, E denotes a state space that is at the most countable, and equipped with the discrete topology. We refer the reader to the definitions and notations of Appendix A.1.

7.1. Preliminaries

We start by stating two technical Lemmas on the exponential distribution, which will be useful in the following.

LEMMA 7.1.– Let U and V be the two independent random variables, of respective distributions ε (λ) and ε (μ), where λ,μ > 0. Then,

images

Proof.

(i) The density of the random couple (U, V) is given for all (u, v) by

images

By denoting the subset A = {(u, v) ∈ R2; uv}, we can write

(ii) It suffices to see that for any

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required