Machine Learning in Java - Second Edition

Evaluation

The submissions were evaluated according to the arithmetic mean of the area under the ROC curve for the three tasks (churn, appetency, and upselling). The ROC curve shows the performance of the model as a curve obtained by plotting the sensitivity against specificity for various threshold values used to determine the classification result (refer to Chapter 1, Applied Machine Learning Quick Start, in the section ROC curves). Now, the area under the ROC curve (AUC) is related to the area under this curve – the larger the area, the better the classifier). Most toolboxes, including Weka, provide an API to calculate the AUC score.