Editing Markov chain (section)

==History==
[[Andrey Markov]] studied Markov processes in the early 20th century, publishing his first paper on the topic in 1906.<ref name="GrinsteadSnell1997page4643">{{cite book|url=https://archive.org/details/flooved3489|title=Introduction to Probability|author1=Charles Miller Grinstead|author2=James Laurie Snell|publisher=American Mathematical Soc.|year=1997|isbn=978-0-8218-0749-1|pages=[https://archive.org/details/flooved3489/page/n473 464]–466}}</ref><ref name="Bremaud2013pageIX3">{{cite book|url=https://books.google.com/books?id=jrPVBwAAQBAJ|title=Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues|author=Pierre Bremaud|date=9 March 2013|publisher=Springer Science & Business Media|isbn=978-1-4757-3124-8|page=ix}}</ref><ref name="Hayes20133">{{cite journal|last1=Hayes|first1=Brian|year=2013|title=First links in the Markov chain|journal=American Scientist|volume=101|issue=2|pages=92–96|doi=10.1511/2013.101.92}}</ref> Markov Processes in continuous time were discovered long before his work in the early 20th century in the form of the [[Poisson point process|Poisson process]].<ref name="Ross1996page235and3583">{{cite book|url=https://books.google.com/books?id=ImUPAQAAMAAJ|title=Stochastic processes|author=Sheldon M. Ross|publisher=Wiley|year=1996|isbn=978-0-471-12062-9|pages=235 and 358}}</ref><ref name="JarrowProtter20042">{{cite book |title= A Festschrift for Herman Rubin |last1=Jarrow |first1=Robert |last2=Protter |first2=Philip |year=2004 |isbn=978-0-940600-61-4 |pages=75–91 |citeseerx=10.1.1.114.632 |doi=10.1214/lnms/1196285381 |chapter=A short history of stochastic integration and mathematical finance: The early years, 1880–1970}}</ref><ref name="GuttorpThorarinsdottir20122">{{cite journal|last1=Guttorp|first1=Peter|last2=Thorarinsdottir|first2=Thordis L.|year=2012|title=What Happened to Discrete Chaos, the Quenouille Process, and the Sharp Markov Property? Some History of Stochastic Point Processes |journal=International Statistical Review|volume=80|issue=2|pages=253–268|doi=10.1111/j.1751-5823.2012.00181.x }}</ref> Markov was interested in studying an extension of independent random sequences, motivated by a disagreement with [[Pavel Nekrasov]] who claimed independence was necessary for the [[weak law of large numbers]] to hold.<ref name="Seneta19962">{{cite journal |year=1996 |title=Markov and the Birth of Chain Dependence Theory |journal=International Statistical Review |volume=64 |issue=3 |pages=255–257 |doi=10.2307/1403785 |author1-link=Eugene Seneta |last1=Seneta |first1=E. |jstor=1403785 }}</ref> In his first paper on Markov chains, published in 1906, Markov showed that under certain conditions the average outcomes of the Markov chain would converge to a fixed vector of values, so proving a weak law of large numbers without the independence assumption,<ref name="GrinsteadSnell1997page4643" /><ref name="Bremaud2013pageIX3" /><ref name="Hayes20133" /> which had been commonly regarded as a requirement for such mathematical laws to hold.<ref name="Hayes20133" /> Markov later used Markov chains to study the distribution of vowels in [[Eugene Onegin]], written by [[Alexander Pushkin]], and proved a [[Markov chain central limit theorem|central limit theorem]] for such chains.<ref name="GrinsteadSnell1997page4643" />

In 1912 [[Henri Poincaré]] studied Markov chains on [[finite group]]s with an aim to study card shuffling. Other early uses of Markov chains include a diffusion model, introduced by [[Paul Ehrenfest|Paul]] and [[Tatyana Ehrenfest]] in 1907, and a branching process, introduced by [[Francis Galton]] and [[Henry William Watson]] in 1873, preceding the work of Markov.<ref name="GrinsteadSnell1997page4643" /><ref name="Bremaud2013pageIX3" /> After the work of Galton and Watson, it was later revealed that their branching process had been independently discovered and studied around three decades earlier by [[Irénée-Jules Bienaymé]].<ref name="Seneta19982">{{cite journal|year=1998|title=I.J. Bienaymé [1796–1878]: Criticality, Inequality, and Internationalization |journal=International Statistical Review |volume=66|issue=3|pages=291–292|doi=10.2307/1403518 |last1=Seneta |first1=E. |jstor=1403518}}</ref> Starting in 1928, [[Maurice Fréchet]] became interested in Markov chains, eventually resulting in him publishing in 1938 a detailed study on Markov chains.<ref name="GrinsteadSnell1997page4643" /><ref name="BruHertz20012">{{cite book |vauthors= Bru B, Hertz S |date= 2001 |chapter= Maurice Fréchet |editor1-link=Chris Heyde |veditors= Heyde CC, Seneta E, Crépel P, Fienberg SE, Gani J |title= Statisticians of the Centuries |publisher= Springer |location= New York, NY |pages= 331–334 |doi= 10.1007/978-1-4613-0179-0_71 |isbn= 978-0-387-95283-3}}</ref>

[[Andrei Kolmogorov|Andrey Kolmogorov]] developed in a 1931 paper a large part of the early theory of continuous-time Markov processes.<ref name="KendallBatchelor1990page332">{{cite journal |last2=Batchelor |first2=G. K. |last3=Bingham |first3=N. H. |last4=Hayman |first4=W. K. |last5=Hyland |first5=J. M. E. |last6=Lorentz |first6=G. G. |last7=Moffatt |first7=H. K. |last8=Parry |first8=W. |last9=Razborov |first9=A. A. |year=1990 |title=Andrei Nikolaevich Kolmogorov (1903–1987) |journal=Bulletin of the London Mathematical Society |volume=22 |issue=1 |page=33 |doi=10.1112/blms/22.1.31 |last1=Kendall |first1=D. G. |last10=Robinson |first10=C. A. |last11=Whittle |first11=P.}}</ref><ref name="Cramer19762">{{cite journal |year=1976 |title=Half a Century with Probability Theory: Some Personal Recollections |journal=The Annals of Probability |volume=4 |issue=4 |pages=509–546 |doi=10.1214/aop/1176996025 |last1=Cramér |first1=Harald |doi-access=free}}</ref> Kolmogorov was partly inspired by Louis Bachelier's 1900 work on fluctuations in the stock market as well as [[Norbert Wiener]]'s work on Einstein's model of Brownian movement.<ref name="KendallBatchelor1990page332" /><ref name="BarbutLocker2016page52">{{cite book |url=https://books.google.com/books?id=lSz_vQAACAAJ |title=Paul Lévy and Maurice Fréchet: 50 Years of Correspondence in 107 Letters |date=23 August 2016 |publisher=Springer London |isbn=978-1-4471-7262-8 |page=5 |author1=Marc Barbut |author2=Bernard Locker |author3=Laurent Mazliak }}</ref> He introduced and studied a particular set of Markov processes known as diffusion processes, where he derived a set of differential equations describing the processes.<ref name="KendallBatchelor1990page332" /><ref name="Skorokhod2005page1462">{{cite book |url=https://books.google.com/books?id=dQkYMjRK3fYC |title=Basic Principles and Applications of Probability Theory |date=5 December 2005 |publisher=Springer Science & Business Media |isbn=978-3-540-26312-8 |page=146 |author=Valeriy Skorokhod }}</ref> Independent of Kolmogorov's work, [[Sydney Chapman (mathematician) |Sydney Chapman]] derived in a 1928 paper an equation, now called the [[Chapman–Kolmogorov equation]], in a less mathematically rigorous way than Kolmogorov, while studying Brownian movement.<ref name="Bernstein20052">{{cite journal |year=2005 |title=Bachelier |journal=American Journal of Physics |volume=73 |issue=5 |pages=395–398 |doi=10.1119/1.1848117 |last1=Bernstein |first1=Jeremy |bibcode=2005AmJPh..73..395B}}</ref> The differential equations are now called the Kolmogorov equations<ref name="Anderson2012pageVII2">{{cite book|url=https://books.google.com/books?id=YpHfBwAAQBAJ&pg=PR8|title=Continuous-Time Markov Chains: An Applications-Oriented Approach|date=6 December 2012|publisher=Springer Science & Business Media|isbn=978-1-4612-3038-0|page=vii|author=William J. Anderson }}</ref> or the Kolmogorov–Chapman equations.<ref name="KendallBatchelor1990page572">{{cite journal |last2=Batchelor |first2=G. K. |last3=Bingham |first3=N. H. |last4=Hayman |first4=W. K. |last5=Hyland |first5=J. M. E. |last6=Lorentz |first6=G. G. |last7=Moffatt |first7=H. K. |last8=Parry |first8=W. |last9=Razborov |first9=A. A. |year=1990 |title=Andrei Nikolaevich Kolmogorov (1903–1987) |journal=Bulletin of the London Mathematical Society |volume=22 |issue=1 |page=57 |doi=10.1112/blms/22.1.31 |last1=Kendall|first1=D. G.|last10=Robinson|first10=C. A.|last11=Whittle|first11=P.}}</ref> Other mathematicians who contributed significantly to the foundations of Markov processes include [[William Feller]], starting in 1930s, and then later [[Eugene Dynkin]], starting in the 1950s.<ref name="Cramer19762" />