In this chapter, we are going to present two probabilistic generative models that employ a set of latent variables to represent a specific data generation process. Restricted Boltzmann Machines (RBMs), proposed in 1986, are the building blocks of a more complex model, called a Deep Belief Network (DBN), which is capable of capturing complex relationships among features at different levels (in a way not dissimilar to a deep convolutional network). Both models can be used in unsupervised and supervised scenarios as preprocessors or, as is usual with DBN, fine-tuning the parameters using a standard backpropagation algorithm.
In particular, we will discuss:
- Markov random fields (MRF)
- RBM
- Contrastive Divergence (CD-k) algorithm ...
