The following graph compares AUC for three different models:
- No regularization
- Mild regularization (10^-6)
- Aggressive regularization (10^-2)
We notice that there is no significant difference between having no regularization and having mild regularization. Aggressive regularization, however, has a direct impact on the model performance. The algorithm converges to a lower AUC, and the optimal learning rate is no longer 100 but 1.
Comparing the performance graph given by Amazon ML for mild and aggressive regularization, we see that although the scores (AUC, accuracy, and so on) are very similar in both cases, the difference ...
