Automatic Differentiation

Automatic differentiation is a computational technique for evaluating the derivatives of functions defined by computer programs, by applying the chain rule repeatedly to the elementary operations the program performs. It differs from symbolic differentiation, which can produce unwieldy expressions, and from numerical differentiation using finite differences, which introduces round-off error; automatic differentiation instead computes derivatives to working precision without either drawback, making it neither a purely numeric nor a purely symbolic method. It is central to training neural networks through backpropagation, and is also used in non-linear optimization, sensitivity analysis, scientific computing, robotics, computer graphics, computer vision and financial modeling, particularly for functions of many variables where computing gradients by hand becomes impractical. This description is adapted from Wikipedia contributors under CC BY-SA 4.0; changes were made. https://creativecommons.org/licenses/by-sa/4.0/