Backpropagation Algorithm

Backpropagation is the algorithm used to train an artificial neural network by computing how much each connection weight in the network contributed to the network's error on a given training example, so the weights can be adjusted to reduce that error. It applies the calculus chain rule working backward through the network's layers, from the output layer toward the input layer, computing the error gradient one layer at a time and reusing each layer's result in the layer before it rather than recalculating it from scratch, which is what makes the method efficient enough to train networks with many layers. The idea has multiple independent roots reaching back to chain-rule and optimal-control mathematics from the 1950s and 1960s; Frank Rosenblatt described an early version he called back-propagating error correction in 1962, though it took further decades of refinement and greater computing power before backpropagation became the practical, near-universal method for training neural networks that it is today.