Editing Expectation–maximization algorithm (section)

== History ==
The EM algorithm was explained and given its name in a classic 1977 paper by [[Arthur P. Dempster|Arthur Dempster]], [[Nan Laird]], and [[Donald Rubin]].<ref name="Dempster1977">
{{cite journal
 |last1=Dempster  |first1= A.P. |author-link1=Arthur P. Dempster
 |last2=Laird |first2=N.M. |author-link2=Nan Laird |last3=Rubin
 |first3=D.B. |author-link3=Donald Rubin
 |title=Maximum Likelihood from Incomplete Data via the EM Algorithm
 |journal=[[Journal of the Royal Statistical Society, Series B]]
 |year=1977 |volume=39 |issue=1 |pages=1–38
 |doi= 10.1111/j.2517-6161.1977.tb01600.x |jstor=2984875 |mr= 0501537
}}
</ref> They pointed out that the method had been "proposed many times in special circumstances" by earlier authors. One of the earliest is the gene-counting method for estimating allele frequencies by [[Cedric Smith (statistician)|Cedric Smith]].<ref>{{cite journal |last1=Ceppelini |first1=R.M. |title=The estimation of gene frequencies in a random-mating population |journal=Ann. Hum. Genet. |volume=20 |issue=2 |pages=97–115 |doi=10.1111/j.1469-1809.1955.tb01360.x|pmid=13268982 |year=1955 |s2cid=38625779 }}</ref> Another was proposed by [[Herman Otto Hartley|H.O. Hartley]] in 1958, and Hartley and Hocking in 1977, from which many of the ideas in the Dempster–Laird–Rubin paper originated.<ref>{{Cite journal |last=Hartley |first=Herman Otto |date=1958 |title=Maximum Likelihood estimation from incomplete data |journal=Biometrics |volume=14 |issue=2 |pages=174–194|doi=10.2307/2527783 |jstor=2527783 }}</ref> Another one by S.K Ng, Thriyambakam Krishnan and G.J McLachlan in 1977.<ref>{{Citation |last1=Ng |first1=Shu Kay |title=The EM Algorithm |date=2011-12-21 |url=http://dx.doi.org/10.1007/978-3-642-21551-3_6 |work=Handbook of Computational Statistics |pages=139–172 |place=Berlin, Heidelberg |publisher=Springer Berlin Heidelberg |isbn=978-3-642-21550-6 |access-date=2022-10-15 |last2=Krishnan |first2=Thriyambakam |last3=McLachlan |first3=Geoffrey J.|doi=10.1007/978-3-642-21551-3_6 |s2cid=59942212 }}</ref> Hartley’s ideas can be broadened to any grouped discrete distribution. A very detailed treatment of the EM method for exponential families was published by Rolf Sundberg in his thesis and several papers,<ref name="Sundberg1974">{{cite journal
 |last=Sundberg |first=Rolf
 |title=Maximum likelihood theory for incomplete data from an exponential family
 |journal=Scandinavian Journal of Statistics
 |volume=1 |year=1974 |issue=2 |pages=49–58
 |jstor=4615553 |mr= 381110
}}</ref><ref name="Sundberg1971">
Rolf Sundberg. 1971. ''Maximum likelihood theory and applications for distributions generated when observing a function of an exponential family variable''. Dissertation, Institute for Mathematical Statistics, Stockholm University.</ref><ref name="Sundberg1976">{{cite journal
 |last=Sundberg |first=Rolf
 |year=1976
 |title=An iterative method for solution of the likelihood equations for incomplete data from exponential families
 |journal=Communications in Statistics – Simulation and Computation
 |volume=5 |issue=1 |pages=55–64
 |doi=10.1080/03610917608812007 |mr=443190
}}</ref> following his collaboration with [[Per Martin-Löf]] and [[Anders Martin-Löf]].<ref>See the acknowledgement by Dempster, Laird and Rubin on pages 3, 5 and 11.</ref><ref name="Martin-Löf1966">[[Per Martin-Löf]]. 1966. ''Statistics from the point of view of statistical mechanics''. Lecture notes, Mathematical Institute, Aarhus University. ("Sundberg formula", credited to Anders Martin-Löf).</ref><ref name="Martin-Löf1970">[[Per Martin-Löf]]. 1970. ''Statistiska Modeller (Statistical Models): Anteckningar från seminarier läsåret 1969–1970 (Lecture notes 1969-1970), with the assistance of Rolf Sundberg.'' Stockholm University. </ref><!-- * Martin-Löf, P. "Exact tests, confidence regions and estimates", with a discussion by [[A. W. F. Edwards]], [[George A. Barnard|G. A. Barnard]], D. A. Sprott, O. Barndorff-Nielsen, [[D. Basu]] and [[Rasch model|G. Rasch]]. ''Proceedings of Conference on Foundational Questions in Statistical Inference'' (Aarhus, 1973), pp. 121–138. Memoirs, No. 1, Dept. Theoret. Statist., Inst. Math., Univ. Aarhus, Aarhus, 1974. --><ref name="Martin-Löf1974a">Martin-Löf, P. The notion of redundancy and its use as a quantitative measure of the deviation between a statistical hypothesis and a set of observational data. With a discussion by F. Abildgård, [[Arthur P. Dempster|A. P. Dempster]], [[D. Basu]], [[D. R. Cox]], [[A. W. F. Edwards]], D. A. Sprott, [[George A. Barnard|G. A. Barnard]], O. Barndorff-Nielsen, J. D. Kalbfleisch and [[Rasch model|G. Rasch]] and a reply by the author. ''Proceedings of Conference on Foundational Questions in Statistical Inference'' (Aarhus, 1973), pp. 1–42. Memoirs, No. 1, Dept. Theoret. Statist., Inst. Math., Univ. Aarhus, Aarhus, 1974.</ref><ref name="Martin-Löf1974b">{{cite journal |last=Martin-Löf |first=Per |title=The notion of redundancy and its use as a quantitative measure of the discrepancy between a statistical hypothesis and a set of observational data |journal=Scand. J. Statist. |volume=1 |year=1974 |issue=1 |pages=3–18 }}</ref> The Dempster–Laird–Rubin paper in 1977 generalized the method and sketched a convergence analysis for a wider class of problems. The Dempster–Laird–Rubin paper established the EM method as an important tool of statistical analysis. See also Meng and van Dyk (1997).

The convergence analysis of the Dempster–Laird–Rubin algorithm was flawed and a correct convergence analysis was published by [[C. F. Jeff Wu]] in 1983.<ref name="Wu">
{{cite journal
|first=C. F. Jeff
|last=Wu
|title=On the Convergence Properties of the EM Algorithm
|journal=[[Annals of Statistics]]
|volume=11
|issue=1
|date=Mar 1983
|pages=95–103
|jstor=2240463
|doi=10.1214/aos/1176346060
|mr= 684867
|doi-access=free
}}</ref>
Wu's proof established the EM method's convergence also outside of the [[exponential family]], as claimed by Dempster–Laird–Rubin.<ref name="Wu" />