Editing Least squares (section)

==History==

===Founding===
The method of least squares grew out of the fields of [[astronomy]] and [[geodesy]], as scientists and mathematicians sought to provide solutions to the challenges of navigating the Earth's oceans during the [[Age of Discovery]]. The accurate description of the behavior of celestial bodies was the key to enabling ships to sail in open seas, where sailors could no longer rely on land sightings for navigation.

The method was the culmination of several advances that took place during the course of the eighteenth century:<ref name=stigler>{{cite book
  | last = Stigler
  | first = Stephen M.
  | title = The History of Statistics: The Measurement of Uncertainty Before 1900
  | publisher = Belknap Press of Harvard University Press
  | year = 1986
  | location = Cambridge, MA
  | isbn = 978-0-674-40340-6
  | url-access = registration
  | url = https://archive.org/details/historyofstatist00stig
  }}</ref>

*The combination of different observations as being the best estimate of the true value; errors decrease with aggregation rather than increase, first appeared in [[Isaac Newton]]'s work in 1671, though it went unpublished, and again in 1700.<ref>{{Cite book |last1=Buchwald |first1=Jed Z. |title=Newton and the Origin of Civilization |last2=Feingold |first2=Mordechai |date=2013 |publisher=Princeton University Press |isbn=978-0-691-15478-7 |location=Princeton Oxford |pages=90–93, 101–103}}</ref><ref>{{Cite web |last=Drum |first=Kevin |date=2013-05-10 |title=The Groundbreaking Isaac Newton Invention You've Never Heard Of |url=https://www.motherjones.com/kevin-drum/2013/05/groundbreaking-isaac-newton-invention-youve-never-heard/ |access-date=2024-12-21 |website=Mother Jones |language=en-US}}</ref> It was perhaps first expressed formally by [[Roger Cotes]] in 1722.
*The combination of different observations taken under the ''same'' conditions contrary to simply trying one's best to observe and record a single observation accurately. The approach was known as the method of averages. This approach was notably used by Newton while studying equinoxes in 1700, also writing down the first of the 'normal equations' known from [[ordinary least squares]],<ref>{{cite arXiv |eprint=0810.4948 |class=physics.hist-ph |first1=Ari |last1=Belenkiy |first2=Eduardo Vila |last2=Echague |title=Groping Toward Linear Regression Analysis: Newton's Analysis of Hipparchus' Equinox Observations |date=2008}}</ref> [[Tobias Mayer]] while studying the [[libration]]s of the Moon in 1750, and by [[Pierre-Simon Laplace]] in his work in explaining the differences in motion of [[Jupiter]] and [[Saturn]] in 1788.
*The combination of different observations taken under {{em|different}} conditions. The method came to be known as the method of ''[[least absolute deviation]]''. It was notably performed by [[Roger Joseph Boscovich]] in his work on the shape of the Earth in 1757 and by [[Pierre-Simon Laplace]] for the same problem in 1789 and 1799.
*The development of a criterion that can be evaluated to determine when the solution with the minimum error has been achieved. Laplace tried to specify a mathematical form of the [[probability]] density for the errors and define a method of estimation that minimizes the error of estimation. For this purpose, Laplace used a symmetric two-sided exponential distribution we now call [[Laplace distribution]] to model the error distribution, and used the sum of absolute deviation as error of estimation. He felt these to be the simplest assumptions he could make, and he had hoped to obtain the arithmetic mean as the best estimate. Instead, his estimator was the posterior median.

===The method===
[[File:Bendixen - Carl Friedrich Gauß, 1828.jpg|thumb|upright=0.8|[[Carl Friedrich Gauss]]]]

The first clear and concise exposition of the method of least squares was published by [[Adrien-Marie Legendre|Legendre]] in 1805.<ref>{{Citation |first=Adrien-Marie |last=Legendre |title=Nouvelles méthodes pour la détermination des orbites des comètes |trans-title=New Methods for the Determination of the Orbits of Comets |language=fr |publisher=F. Didot |location=Paris |year=1805 |url=https://books.google.com/books?id=FRcOAAAAQAAJ |hdl=2027/nyp.33433069112559 |hdl-access=free }}</ref> The technique is described as an algebraic procedure for fitting linear equations to data and Legendre demonstrates the new method by analyzing the same data as Laplace for the shape of the Earth. Within ten years after Legendre's publication, the method of least squares had been adopted as a standard tool in astronomy and geodesy in [[France]], [[Italy]], and [[Prussia]], which constitutes an extraordinarily rapid acceptance of a scientific technique.<ref name=stigler></ref>

In 1809 [[Carl Friedrich Gauss]] published his method of calculating the orbits of celestial bodies. In that work he claimed to have been in possession of the method of least squares since 1795.<ref>{{Cite web |last= |date=2015-11-06 |title=The Discovery of Statistical Regression |url=https://priceonomics.com/the-discovery-of-statistical-regression/ |access-date=2023-04-04 |website=Priceonomics |language=en}}</ref> This naturally led to a priority dispute with Legendre. However, to Gauss's credit, he went beyond Legendre and succeeded in connecting the method of least squares with the principles of probability and to the [[normal distribution]]. He had managed to complete Laplace's program of specifying a mathematical form of the probability density for the observations, depending on a finite number of unknown parameters, and define a method of estimation that minimizes the error of estimation. Gauss showed that the [[arithmetic mean]] is indeed the best estimate of the location parameter by changing both the [[probability density]] and the method of estimation. He then turned the problem around by asking what form the density should have and what method of estimation should be used to get the arithmetic mean as estimate of the location parameter. In this attempt, he invented the normal distribution.

An early demonstration of the strength of [[Gauss's method]] came when it was used to predict the future location of the newly discovered asteroid [[Ceres (dwarf planet)|Ceres]]. On 1 January 1801, the Italian astronomer [[Giuseppe Piazzi]] discovered Ceres and was able to track its path for 40 days before it was lost in the glare of the Sun. Based on these data, astronomers desired to determine the location of Ceres after it emerged from behind the Sun without solving [[Kepler's laws of planetary motion|Kepler's complicated nonlinear equations]] of planetary motion. The only predictions that successfully allowed Hungarian astronomer [[Franz Xaver von Zach]] to relocate Ceres were those performed by the 24-year-old Gauss using least-squares analysis.

In 1810, after reading Gauss's work, Laplace, after proving the [[central limit theorem]], used it to give a large sample justification for the method of least squares and the normal distribution. In 1822, Gauss was able to state that the least-squares approach to regression analysis is optimal in the sense that in a linear model where the errors have a mean of zero, are uncorrelated, normally distributed, and have equal variances, the best linear unbiased estimator of the coefficients is the least-squares estimator. An extended version of this result is known as the [[Gauss–Markov theorem]].

The idea of least-squares analysis was also independently formulated by the American [[Robert Adrain]] in 1808. In the next two centuries workers in the theory of errors and in statistics found many different ways of implementing least squares.<ref>{{cite journal |doi=10.1111/j.1751-5823.1998.tb00406.x |first=J. |last=Aldrich |year=1998|title=Doing Least Squares: Perspectives from Gauss and Yule |journal=International Statistical Review |volume=66 |issue=1 |pages= 61–81|s2cid=121471194 }}</ref>