Machine Learning

Contrastive divergence

To train the RBM, one has to obtain the positive and negative phase correlations. However, the computation of the latter is intractable. A way to approach it is via Gibbs sampling techniques (see Chapter 14). The fact that we know analytically the conditionals, P(h_i|v) and P(v_j|h), allows us to apply Gibbs sampling by sequentially drawing samples, that is, ${\mathit{h}}^{(1)}\backsim P(\mathit{h}|{\mathit{v}}^{(1)})$, ${\mathit{v}}^{(2)}\backsim P(\mathit{v}|{\mathit{h}}^{(1)})$, ${\mathit{h}}^{(2)}\backsim P(\mathit{h}|{\mathit{v}}^{(2)})$, and so on. However, ...