Combining Pattern Classifiers: Methods and Algorithms, 2nd Edition

3 AN OVERVIEW OF THE FIELD

3.1 PHILOSOPHY

A classifier ensemble is sketched in Figure 3.1a. Several classifiers are employed to make a classification decision about the object submitted at the input, and the individual decisions are subsequently aggregated. The output of the ensemble is a class label for the object.

FIGURE 3.1 What is a classifier ensemble?

Classifier ensembles are justly receiving increasing attention and accolade and generating a wealth of research [53, 183, 311, 321, 335, 355, 357, 397, 425, 439]. Theoretical and empirical studies have demonstrated that an ensemble of classifiers is typically more accurate than a single classifier. Research on classifier ensembles permeate many strands of machine learning including streaming data [160, 326], biometrics [312], concept drift, and incremental learning [119].

Intuitive as this concept may be, there is no rigorous definition of a classifier ensemble. Figures 3.1b–d illustrate the uncertainty of the generic definition. Any classifier ensemble is, in fact, a classifier (Figure 3.1b). We can think of the constituent classifiers (called “base classifiers”) as fancy feature extractors, while the combiner would be a simple classifier that aggregates the “fancy features.” On the other hand, what is stopping us from proclaiming that a standard neural network classifier is a classifier ensemble (Figure 3.1c)? The neurons ...