Newton boosting trumps gradient boosting for better predictive accuracy in data science.
Boosting algorithms are commonly used in data science. This article compares two types of boosting methods, gradient and Newton boosting, as well as a combination of both. The researchers found that Newton boosting performs better in predicting outcomes on most datasets compared to gradient and hybrid boosting. Surprisingly, the reason for this is not because Newton boosting converges faster. The researchers also introduced a new parameter for tree-based Newton boosting that improves predictive accuracy.
