Levenberg-Marquardt Algorithm

In mathematics and computing, the Levenberg-Marquardt algorithm, also known as the damped least-squares method, is used to solve non-linear least squares problems, which arise especially in least squares curve fitting. It interpolates between the Gauss-Newton algorithm and the method of gradient descent, and is more robust than the Gauss-Newton algorithm in that it often finds a solution even starting far from the final minimum, though for well-behaved functions and reasonable starting parameters it tends to be slower. The algorithm was first published in 1944 by Kenneth Levenberg while working at the Frankford Army Arsenal, and was rediscovered in 1963 by Donald Marquardt, a statistician at DuPont, and independently by Girard, Wynne and Morrison. It is used in many software applications for generic curve-fitting problems, converging faster than first-order methods by using the Gauss-Newton algorithm, though like other iterative optimization algorithms it finds only a local minimum, not necessarily the global one.