Introduction to Stochastic Models

Introduction to Stochastic Models

Released April 2010
Publisher(s): Wiley
ISBN: 9781848210578

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

This book provides a pedagogical examination of the way in which stochastic models are encountered in applied sciences and techniques such as physics, engineering, biology and genetics, economics and social sciences. It covers Markov and semi-Markov models, as well as their particular cases: Poisson, renewal processes, branching processes, Ehrenfest models, genetic models, optimal stopping, reliability, reservoir theory, storage models, and queuing systems. Given this comprehensive treatment of the subject, students and researchers in applied sciences, as well as anyone looking for an introduction to stochastic models, will find this title of invaluable use.

Table of contents

  1. Cover
  2. Dedication
  3. Title Page
  4. Copyright
  5. Preface
  6. Chapter 1: Introduction to Stochastic Processes
    1. 1.1. Sequences of random variables
    2. 1.2. The notion of stochastic process
    3. 1.3. Martingales
    4. 1.4. Markov chains
    5. 1.5. State classification
    6. 1.6. Continuous-time Markov processes
    7. 1.7. Semi-Markov processes
  7. Chapter 2: Simple Stochastic Models
    1. 2.1. Urn models
    2. 2.2. Random walks
    3. 2.3. Brownian motion
    4. 2.4. Poisson processes
    5. 2.5. Birth and death processes
  8. Chapter 3: Elements of Markov Modeling
    1. 3.1. Markov models: ideas, history, applications
    2. 3.2. The discrete-time Ehrenfest model
    3. 3.3. Markov models in genetics
    4. 3.4. Markov storage models
    5. 3.5. Reliability of Markov models
  9. Chapter 4: Renewal Models
    1. 4.1. Fundamental concepts and examples
    2. 4.2. Waiting times
    3. 4.3. Modified renewal processes
    4. 4.4. Replacement models
    5. 4.5. Renewal reward processes
    6. 4.6. The risk problem of an insurance company
    7. 4.7. Counter models
    8. 4.8. Alternating renewal processes
    9. 4.9. Superposition of renewal processes
    10. 4.10. Regenerative processes
  10. Chapter 5: Semi-Markov Models
    1. 5.1. Introduction
    2. 5.2. Markov renewal processes
    3. 5.3. First-passage times and state classification
    4. 5.4. Reliability
    5. 5.5. Reservoir models
    6. 5.6. Queues
    7. 5.7. Digital communication channels
  11. Chapter 6: Branching Models
    1. 6.1. The Bienaymé-Galton-Watson model
    2. 6.2. Generalizations of the B-G-W model
    3. 6.3. Continuous-time models
  12. Chapter 7: Optimal Stopping Models
    1. 7.1. The classic optimal stopping problem
    2. 7.2. Renewal with binary decision
  13. Bibliography
  14. Notation
  15. Index

Product information

  • Title: Introduction to Stochastic Models
  • Author(s):
  • Release date: April 2010
  • Publisher(s): Wiley
  • ISBN: 9781848210578