Gradient boosting builds an ensemble of models, typically decision trees, in a sequential stage-wise manner, where each new model is trained to predict the residual errors, expressed as the negative gradient of a chosen loss function, left by the ensemble built so far, and its predictions are added to the ensemble with a scaling factor called the learning rate. Repeating this over many stages gradually reduces the overall loss on the training data. Jerome Friedman formalized the technique in a 1999 paper generalizing earlier boosting methods, and it underlies widely used modern implementations such as XGBoost and LightGBM.
Facts
Partially Attested
Credited ToFriedman developed explicit regression gradient boosting; the source says the idea originated with Leo Breiman and that the functional gradient view was developed simultaneously by Mason, Baxter, Bartlett and Frean Classification
Design Technique Connections
Associated With
Both are the two dominant general ensemble-learning paradigms in the machine-learning literature: bagging combines base learners trained in parallel on bootstrap-resampled data, gradient boosting builds an ensemble sequentially, each stage correcting the previous stage's residual error.
In Field
Source Wikipedia: Gradient boosting
Uses Design Technique
Entity-backed identity for the design-technique enum value this algorithm already carries, resolved to a computing concept by an explicit value-to-entity map (phase 3 bucket conversion, docs\design_entity_backed_browse_buckets_20260928.md). The design-technique fact itself stays on the algorithm unchanged.
Sources
1. Gradient Boosting Algorithm (Wikipedia)
Wikipedia: design technique greedyQuote, Wikipedia: design technique greedy
greedy
2. Wikipedia: Gradient boosting
History
Explicit regression gradient boosting algorithms were subsequently developed, by Jerome H. Friedman
In Field: Machine Learning, Lead sentence
Gradient boosting is a machine learning technique based on boosting in a functional space, where the target is pseudo-residuals instead of residuals as in traditional boosting.
View the SourceReader Challenges (0)
No disputes yet. Spotted an error or a better source? Open the first one.
Sign in to dispute this or suggest a correction.