O'Reilly logo

Bayesian Analysis of Stochastic Process Models by Mike Wiper, Fabrizio Ruggeri, David Insua

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

4

Continuous time Markov chains and extensions

4.1 Introduction

In this chapter, we consider inference, prediction and decision-making tasks with continuous time Markov chains (CTMCs) and some of their extensions. Our interest in such processes is twofold. First, they constitute an extension of discrete time Markov chains, which were dealt with in Chapter 3. Throughout this chapter, we shall use some of the results shown there. Second, CTMCs have many applications, either directly or as basic building blocks in areas such as queueing, reliability analysis, risk analysis, or biomedical applications, some of which are presented in later chapters.

CTMCs are continuous time stochastic processes with discrete state space. We shall concentrate on homogeneous CTMCs with finite state space. In those processes, the system remains an exponential time at each state and, when leaving such state, it evolves according to probabilities that depend only on the leaving state. The basic probabilistic results for CTMCs of this type are outlined in Section 4.2.

The parameters of interest of the CTMC are the transition probabilities and the exponential permanence rates. Given a completely observed CTMC, inference for the transition probabilities can be carried out as in Chapter 3. In Section 4.3, we show how to extend this procedure to consider the CTMC rates and, as a relevant by-product, we deal with inference for the intensity matrix of the process. Short- and long-term forecasting are also considered. ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required