Discrete Stochastic Processes and Optimal Filtering

6.5. Gradient algorithm

We have seen previously that the vector λ optimal, which is to say the one that minimizes the cost is written:

Now, to resolve this equation, we have to inverse the autocorrelation matrix. That can involve major calculations if this matrix R is not a Toeplitz matrix. It is a Toeplitz matrix if R_{(i, j)} = c_(i−i) with c representing the autocorrelation of the process.

Let us examine the evolution of the cost previously traced.

Let λ_K be the vector coefficients (or weight) at instant K. If we wish to arrive at λ optimal, we must make λ_K evolve at each interaction by taking into account its relative position between the instant K and K+1.

For a given cost , the gradient of with regards to the vector is normal at .

In order for the algorithm to converge, it must very obviously ...