Learning as inference
In previous chapters we largely assumed that all distributions are fully specified for the inference tasks. In machine learning and related fields, however, the distributions need to be learned on the basis of data. Learning is then the problem of integrating data with domain knowledge of the model environment. In this chapter we discuss how learning can be phrased as an inference problem.
9.1 Learning as inference
9.1.1 Learning the bias of a coin
Consider data expressing the results of tossing a coin. We write vn = 1 if on toss n the coin comes up heads, and vn = 0 if it is tails. Our aim is to estimate the probability θ that the coin will be a head, p(vn = 1|θ) = θ – called the ‘bias’ of the coin. For a fair coin, ...