Editing Gauss–Markov theorem (section)

{{Short description|Theorem related to ordinary least squares}}
{{distinguish|Gauss–Markov process}} 
{{Redirect|BLUE|queue management algorithm|Blue (queue management algorithm)|the color|Blue}}
{{Regression bar}}

In [[statistics]], the '''Gauss–Markov theorem''' (or simply '''Gauss theorem''' for some authors)<ref>See chapter 7 of {{cite book|author1=Johnson, R.A.|author2=Wichern, D.W.|year=2002|title=Applied multivariate statistical analysis|volume=5|publisher=Prentice hall}}</ref> states that the [[ordinary least squares]] (OLS) estimator has the lowest [[sampling variance]] within the [[Class (set theory)|class]] of [[Linear combination|linear]] [[bias of an estimator|unbiased]] [[estimator]]s, if the [[Errors and residuals|errors]] in the [[linear regression model]] are [[uncorrelated]], have [[Homoscedasticity|equal variances]] and expectation value of zero.<ref>{{cite book |first=Henri |last=Theil |author-link=Henri Theil |chapter=Best Linear Unbiased Estimation and Prediction |title=Principles of Econometrics |url=https://archive.org/details/principlesofecon0000thei |url-access=registration |location=New York |publisher=John Wiley & Sons |year=1971 |pages=[https://archive.org/details/principlesofecon0000thei/page/119 119]–124 |isbn=0-471-85845-5 }}</ref> The errors do not need to be [[normal distribution|normal]], nor do they need to be [[independent and identically distributed]] (only [[uncorrelated]] with mean zero and [[homoscedastic]] with finite variance). The requirement that the estimator be unbiased cannot be dropped, since biased estimators exist with lower variance. See, for example, the [[James–Stein estimator]] (which also drops linearity), [[ridge regression]], or simply any [[Degenerate distribution|degenerate]] estimator.

The theorem was named after [[Carl Friedrich Gauss]] and [[Andrey Markov]], although Gauss' work significantly predates Markov's.<ref>{{cite journal |first=R. L. |last=Plackett |author-link=Robin Plackett |title=A Historical Note on the Method of Least Squares |journal=[[Biometrika]] |volume=36 |issue=3/4 |year=1949 |pages=458–460 |doi=10.2307/2332682 }}</ref> But while Gauss derived the result under the assumption of independence and normality, Markov reduced the assumptions to the form stated above.<ref>{{cite journal |first=F. N. |last=David |first2=J. |last2=Neyman |title=Extension of the Markoff theorem on least squares |journal=Statistical Research Memoirs |year=1938 |volume=2 |pages=105–116 |oclc=4025782 }}</ref> A further generalization to [[Heteroscedasticity|non-spherical errors]] was given by [[Alexander Aitken]].<ref name="Aitken1935" />