With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

No credit card required

The Least Squares Method

In this chapter, we are going to present a series of techniques based on minimizing mean square criteria to solve linear problems. But first we are going to state a fundamental theorem, called the projection theorem. It was mentioned more or less explicitly in the affine trend suppression problem, or when we estimated the amplitudes of a harmonic signal’s components. We will see that it has major applications both in a deterministic or random context.

9.1 The projection theorem

The projection theorem is presented in mathematical form. However, readers that are not used to this formalism should not be worried, since the result expressed by relation 9.1 is quite intuitive, as it is shown in Figure 9.1.

Definition 9.1 (Hilbert space) Let be a vector space with a dot product (x, y) for any two of its elements:

the norm of an element x of is the positive number defined by ||x|| = ;
x and y are said to be orthogonal, which is denoted by xy, if (x, y) = 0;
the distance between two elements x and y of is the positive number defined by d(x, y) = || ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

No credit card required