Markov Chains: Analytic and Monte Carlo Computations

Preface

This book was born from my teaching experience in French engineering schools.In these, mathematical tools should be introduced by showing that they providemodels allowing for exact or approximate computations for relevant quantitiespertaining to realistic phenomena.

I have taught in particular a course on the Markovchains, the theory of which is already old. I had learnt a lot of it by osmosis whilestudying or doing research on more recent topics in stochastics. Teaching it forced medo delve deeper into it. This allowed me to rediscover the power and finesse ofthe probabilistic tools based on the work by Kai Lai Chung,Wolfang (or Vincent) Doeblin, Joseph Leo Doob, William Feller, and AndreiKolmogorov, which laid the ground for stochastic calculus.

I realized that Markovchain theory is actually a very active research field, both for theory and forapplications. The derivation of efficient Monte Carlo algorithms and theirvariants, for instance adaptive ones, is a hot subject, and often these are theonly methods allowing tractable computations for treating the enormousquantities of data that scientists and engineers can now acquire. This neednotably feeds theoretical studies on long-time rates of convergence for Markovchains.

This book is aimed at a public of engineering school and masterstudents and of applied scientists and engineers, wishing to acquire thepertinent mathematical bases. It is structured and animated by a few classicexamples, which are each investigated ...