Markov Processes for Stochastic Modeling, 2nd Edition

4

Discrete-Time Markov Chains

4.1 Introduction

The discrete-time process is called a Markov chain if for all , the following is true:

(4.1)

The quantity is called the state-transition probability, which is the conditional probability that the process will be in state j at time k immediately after the next transition, given that it is in state i at time ...