Gauss-Newton algorithm

The Gauss-Newton algorithm solves non-linear least squares problems, which is the same task as minimizing a sum of squared function values. It extends Newton's method for finding a minimum of a non-linear function. Because a sum of squares can never be negative, the algorithm can be viewed as applying Newton's method to approximate, step by step, the zeroes of the components of the sum, and so to minimize the sum; in this sense it is also an effective way to solve overdetermined systems of equations. Its chief advantage is that it needs no second derivatives, which can be hard to compute. Non-linear least squares problems arise, for instance, in non-linear regression, where the parameters of a model are sought so that the model agrees well with the available observations. The method is named after the mathematicians Carl Friedrich Gauss and Isaac Newton, and it first appeared in Gauss's 1809 work Theoria motus corporum coelestium in sectionibus conicis solem ambientum.