undefi
Reduce Overfitting
- learn a less complex gradient boosted tree model
- reduce the learning rate, so that each tree doesn't try as hard to learn a more complex model
- reduce the max_depth parameter for the individual trees in the ensemble
Decision boundaries
- have the box-like shape that's characteristic of decision trees or ensembles of trees
pros
- Achieve excellent accuracy
- once the model has been trained, only require modest memory and runtime to perform prediction
- using the model to make prediction only requires modest memory and is fast
- doesn't require careful features normalization to perform well.
- handle a mixture of feature types. (binary, continuous, categorical types)
cons
- the models are difficult for people to interpret.
- requires careful tuning of the learning rate and other parameters.
- the training process can require significant computation
- like decision trees, not recommended for text classification and other problems with very high dimensional sparse features, for accuracy and computational cost reasons.