O'Reilly logo

Ensemble Methods in Data Mining by John Elder, Giovanni Seni

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

CHAPTER 6

Ensemble Complexity

Ensemble1 models appear very complex, yet we have seen how they can strongly outperform their component models on new data. This seems to violate “Occam’s Razor” – the useful and widespread analytic doctrine that can be stated “when accuracy of two hypotheses is similar, prefer the simpler.” We argue that the problem is really that complexity has traditionally been measured incorrectly. Instead of counting parameters to assess the complexity of a modeling process (as with linear regression), we need to instead measure its flexibility – as by Generalized Degrees of Freedom, GDF (Ye, J., 1998). By measuring complexity according to a model’s behavior rather than its appearance, the utility of Occam’s Razor is restored. ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required