Let's look at this in action by taking three simple models as an example. As you can see, the ground truth is all 1s:
1111111100 = 80% accuracy 1111111100 = 80% accuracy 1011111100 = 70% accuracy
These models are highly correlated in their predictions. When we take a majority vote, we see no improvement:
1111111100 = 80% accuracy
Now, let's compare that to the following three lower-performing but highly uncorrelated models:
1111111100 = 80% accuracy 0111011101 = 70% accuracy 1000101111 = 60% accuracy
When we ensemble this with a majority vote, we get the following result:
1111111101 = 90% accuracy
Here, we see a much higher rate of improvement than in any of our individual models. Low correlation between ...
