Introduction to Stochastic Models

Chapter 7

Optimal Stopping Models

Some optimal stopping problems, such as, for instance, the secretary problem, have a long history. The optimal stopping theory started to form a specific branch of the stochastic processes theory during the 40s. In the period between the publication of A. Wald’s monograph (1947) and that of A. N. Shiryaev (1978), we witness an increasing interest in solving certain specific problems as well as in building up an ever more comprising general theory.

In this chapter we will describe the general issues of this theory as well as some classic models.

7.1. The classic optimal stopping problem

7.1.1. Formulation of the problem

First of all we will give some examples.

EXAMPLE 7.1.– Let us consider a repeated coin tossing, with the convention that player A earns one unit if tails occurs, whereas player B gets one unit if heads occur. If S_n is the gain of A after n rounds, then

where is a sequence of i.i.d. r.v. with .¹ We have described up to now the probabilistic structure of the experiment and now we will specify the structure of the decision. We first assume that the amount of money that the two players have at their disposal is unlimited. What they both ...