An iterative optimization algorithm that repeatedly adjusts a set of parameters in the direction that most decreases a target function, moving downhill along the function's gradient until it settles near a minimum; the core optimizer behind training most machine-learning models.
Facts
Core PrincipleGradient descent is an iterative optimization method that takes repeated steps in the direction opposite the gradient of a function at the current point, because that is the direction of steepest descent, moving the estimate toward a local minimum. 1 Connections
Sources
1. Wikipedia: Gradient Descent
Wikimedia FoundationDescription section, steepest descent
The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point, because this is the direction of steepest descent.
History section, Cauchy attribution
Gradient descent is generally attributed to Augustin-Louis Cauchy, who first suggested it in 1847.
View the Source Reader Challenges (0)
No disputes yet. Spotted an error or a better source? Open the first one.
Sign in to dispute this or suggest a correction.