Probability, Random Variables, and Random Processes: Theory and Signal Processing Applications

7.3 LAWS OF LARGE NUMBERS

The laws of large numbers describe how a specific function of a random sequence behaves when the time argument approaches infinity. They are special cases of the previous convergence definitions regarding convergence of the sample mean to a random variable or a constant. Consider the iid random sequence X[k] with mean and finite variance . Define the following sample mean which itself is a random sequence:

(7.41)

and is an estimator of the ensemble mean (estimators and their properties are discussed in Chapter 9).

Theorem 7.5 (Weak law of large numbers). The sample mean estimator converges in probability to :

(7.42)

for every .

Proof. Since X[k] is assumed to be ...