Editing Automatic differentiation (section)

{{Short description|Numerical calculations carrying along derivatives}}
In [[mathematics]] and [[computer algebra]], '''automatic differentiation''' ('''auto-differentiation''', '''autodiff''', or '''AD'''), also called '''algorithmic differentiation''', '''computational differentiation''', and '''differentiation arithmetic'''<ref>{{cite journal|last=Neidinger|first=Richard D.|title=Introduction to Automatic Differentiation and MATLAB Object-Oriented Programming|journal=SIAM Review| year=2010| volume=52| issue=3| pages=545–563| url=http://academics.davidson.edu/math/neidinger/SIAMRev74362.pdf|doi=10.1137/080743627| citeseerx=10.1.1.362.6580|s2cid=17134969 }}</ref><ref name="baydin2018automatic">{{cite journal|last1=Baydin|first1=Atilim Gunes|last2=Pearlmutter| first2=Barak|last3=Radul|first3=Alexey Andreyevich|last4=Siskind|first4=Jeffrey|title=Automatic differentiation in machine learning: a survey| journal=Journal of Machine Learning Research|year=2018|volume=18|pages=1–43|url=http://jmlr.org/papers/v18/17-468.html}}</ref><ref name="Dawood.Megahed.2023">[[Hend Dawood]] and [[Nefertiti Megahed]] (2023). Automatic differentiation of uncertainties: an interval computational differentiation for first and higher derivatives with implementation. PeerJ Computer Science 9:e1301 https://doi.org/10.7717/peerj-cs.1301.</ref><ref name="Dawood.Megahed.2019">[[Hend Dawood]] and [[Nefertiti Megahed]] (2019). A Consistent and Categorical Axiomatization of Differentiation Arithmetic Applicable to First and Higher Order Derivatives. Punjab University Journal of Mathematics. 51(11). pp. 77-100. doi: 10.5281/zenodo.3479546. http://doi.org/10.5281/zenodo.3479546.</ref> is a set of techniques to evaluate the [[partial derivative]] of a function specified by a computer program. Automatic differentiation is a subtle and central tool to automatize the simultaneous computation of the numerical values of arbitrarily complex functions and their derivatives with no need for the symbolic representation of the derivative, only the function rule or an algorithm thereof is required.<ref name="Dawood.Megahed.2023"/><ref name="Dawood.Megahed.2019"/> Auto-differentiation is thus neither numeric nor symbolic, nor is it a combination of both. It is also preferable to ordinary numerical methods: In contrast to the more traditional numerical methods based on finite differences, auto-differentiation is 'in theory' exact, and in comparison to symbolic algorithms, it is computationally inexpensive.<ref name="Dawood-attribution">{{Creative Commons text attribution notice|cc=bysa4|url=https://pmc.ncbi.nlm.nih.gov/articles/PMC10280627/|author(s)=Dawood and Megahed}}</ref><ref name="Dawood.Megahed.2023"/><ref name="Dawood.Dawood.2022">[[Hend Dawood]] and [[Yasser Dawood]] (2022). Interval Root Finding and Interval Polynomials: Methods and Applications in Science and Engineering. In S. Chakraverty, editor, Polynomial Paradigms: Trends and Applications in Science and Engineering, chapter 15. IOP Publishing. ISBN 978-0-7503-5065-5. doi: 10.1088/978-0-7503-5067-9ch15. URL https://doi.org/10.1088/978-0-7503-5067-9ch15.</ref>

Automatic differentiation exploits the fact that every computer calculation, no matter how complicated, executes a sequence of elementary arithmetic operations (addition, subtraction, multiplication, division, etc.) and elementary functions ([[exponential function|exp]], [[natural logarithm|log]], [[sine|sin]], [[cosine|cos]], etc.). By applying the [[chain rule]] repeatedly to these operations, partial derivatives of arbitrary order can be computed automatically, accurately to working precision, and using at most a small constant factor of more arithmetic operations than the original program.