The simplest ensembling technique is perhaps to take a simple majority. This works on the intuition that a single model might make an error on a particular prediction but that several different models are unlikely to make identical errors.
Let's look at an example.
Ground truth: 11011001
The numbers 1 and 0 represent a True and False prediction for an imagined binary classifier. Each digit is a single true or false prediction for different inputs.
Let's assume there are three models with only one error for this example; they are as follows:
- Model A prediction: 10011001
- Model B prediction: 11011001
- Model C prediction: 11011001
The majority votes gives us the correct answer as follows: ...
