Introduction to Stochastic Models

Preface

Stochastic models have become a very useful tool, indeed fundamental, for a lot of sciences and applied engineering. Whether it be in theoretical or applied physics, economy or social sciences, engineering or even music, stochastic models are there to help.

Stochastic modeling is mainly based on Markov processes, characterized by the memoryless property, conditioned on their position at present. This is the probabilistic analogue of dynamical systems, where the future position of a moving particle depends only on the current position and speed. The natural generalization of Markov processes to semi-Markov processes offers much more generality, as the sojourn time in a state can have any arbitrary distribution on the real half-line x ≥ 0, and not only an exponential distribution as in the Markovian case. Let us also notice that martingales are increasingly used for real system modeling, especially in financial mathematics and statistics.

This book is based on the book [IOS 84] published in 1984, in Romanian. We will follow the main lines of that book, but will replace quite a lot of the material. We would like to pay tribute to the memory of our co-authors of [IOS 84], Serban Grigorescu (1946–1997) and Gheorghe Popescu (1944–1989), who unfortunately died before fulfilling their creative potential.

Throughout this book, composed of seven chapters, we will focus on Markov and semi-Markov processes and on their particular cases: Poisson and renewal processes.

The aim of our ...