Evaluating binary classification

Once your model is ready, click on the model's title from the service dashboard to access the model's result page, which contains the summary of the model, its settings and the evaluation results.

The following screenshot shows that we obtained an AUC score of 0.880, which is considered very good for most machine-learning applications. AUC stands for the Area under the Curve and was introduced in Chapter 2Machine Learning Definitions and Concepts. It is the de-facto metric for classification problems.

The baseline for Binary classification is an AUC of 0.5, which is the score for a model that would randomly predict 0 or 1:

Amazon ML validates the model by checking the following conditions and raising alerts ...

Get Effective Amazon Machine Learning now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.