O'Reilly logo

Ensemble Machine Learning by Ankit Dixit

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Tune the number of decision trees in XGBoost

Most implementations of gradient boosting are configured by default with a relatively small number of trees, such as hundreds or thousands. The general reason is that on most problems, adding more trees beyond a limit does not improve the performance of the model.

The reason is in the way the boosted tree model is constructed—sequentially—where each new tree attempts to model and correct the errors made by the sequence of the previous trees. Quickly, the model reaches a point of diminishing returns.

We can demonstrate this point of diminishing returns easily on the Otto dataset. The number of trees (or rounds) in an XGBoost model is specified to the XGBClassifier or XGBRegressor class in the

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required