Editing Stochastic process (section)

==History==

===Early probability theory===
Probability theory has its origins in games of chance, which have a long history, with some games being played thousands of years ago,<ref name="David1955">{{cite journal|last1=David|first1=F. N.|title=Studies in the History of Probability and Statistics I. Dicing and Gaming (A Note on the History of Probability)|journal=Biometrika|volume=42|issue=1/2|pages=1–15|year=1955|issn=0006-3444|doi=10.2307/2333419|jstor=2333419}}</ref> but very little analysis on them was done in terms of probability.<ref name="Maistrov2014page1">{{cite book|author=L. E. Maistrov|title=Probability Theory: A Historical Sketch|url=https://books.google.com/books?id=2ZbiBQAAQBAJ&pg=PR9|year=2014|publisher=Elsevier Science|isbn=978-1-4832-1863-2|page=1}}</ref> The year 1654 is often considered the birth of probability theory when French mathematicians [[Pierre Fermat]] and [[Blaise Pascal]] had a written correspondence on probability, motivated by a [[Problem of points|gambling problem]].<ref name="Seneta2006page1">{{cite book|author1-link=Eugene Seneta|last1=Seneta|first1=E.|title=Encyclopedia of Statistical Sciences|chapter=Probability, History of|year=2006|doi=10.1002/0471667196.ess2065.pub2|page=1|isbn=978-0471667193}}</ref><ref name="Tabak2014page24to26">{{cite book|author=John Tabak|title=Probability and Statistics: The Science of Uncertainty|url=https://books.google.com/books?id=h3WVqBPHboAC|year=2014|publisher=Infobase Publishing|isbn=978-0-8160-6873-9|pages=24–26}}</ref> But there was earlier mathematical work done on the probability of gambling games such as ''Liber de Ludo Aleae'' by [[Gerolamo Cardano]], written in the 16th century but posthumously published later in 1663.<ref name="Bellhouse2005">{{cite journal|last1=Bellhouse|first1=David|title=Decoding Cardano's Liber de Ludo Aleae|journal=Historia Mathematica|volume=32|issue=2|year=2005|pages=180–202|issn=0315-0860|doi=10.1016/j.hm.2004.04.001|doi-access=free}}</ref>

After Cardano, [[Jakob Bernoulli]]{{efn|Also known as James or Jacques Bernoulli.<ref name="Hald2005page221">{{cite book|author=Anders Hald|title=A History of Probability and Statistics and Their Applications before 1750|url=https://books.google.com/books?id=pOQy6-qnVx8C|year=2005|publisher=John Wiley & Sons|isbn=978-0-471-72517-6|page=221}}</ref>}} wrote [[Ars Conjectandi]], which is considered a significant event in the history of probability theory. Bernoulli's book was published, also posthumously, in 1713 and inspired many mathematicians to study probability.<ref name="Maistrov2014page56">{{cite book|author=L. E. Maistrov|title=Probability Theory: A Historical Sketch|url=https://books.google.com/books?id=2ZbiBQAAQBAJ&pg=PR9|year=2014|publisher=Elsevier Science|isbn=978-1-4832-1863-2|page=56}}</ref><ref name="Tabak2014page37">{{cite book|author=John Tabak|title=Probability and Statistics: The Science of Uncertainty|url=https://books.google.com/books?id=h3WVqBPHboAC|year=2014|publisher=Infobase Publishing|isbn=978-0-8160-6873-9|page=37}}</ref> But despite some renowned mathematicians contributing to probability theory, such as [[Pierre-Simon Laplace]], [[Abraham de Moivre]], [[Carl Gauss]], [[Siméon Poisson]] and [[Pafnuty Chebyshev]],<ref name="Chung1998">{{cite journal|last1=Chung|first1=Kai Lai|title=Probability and Doob|journal=The American Mathematical Monthly|volume=105|issue=1|pages=28–35|year=1998|issn=0002-9890|doi=10.2307/2589523|jstor=2589523}}</ref><ref name="Bingham2000">{{cite journal|last1=Bingham|first1=N.|title=Studies in the history of probability and statistics XLVI. Measure into probability: from Lebesgue to Kolmogorov|journal=Biometrika|volume=87|issue=1|year=2000|pages=145–156|issn=0006-3444|doi=10.1093/biomet/87.1.145}}</ref> most of the mathematical community{{efn|It has been remarked that a notable exception was the St Petersburg School in Russia, where mathematicians  led by Chebyshev studied probability theory.<ref name="BenziBenzi2007">{{cite journal|last1=Benzi|first1=Margherita|last2=Benzi|first2=Michele|last3=Seneta|first3=Eugene|title=Francesco Paolo Cantelli. b. 20 December 1875 d. 21 July 1966|journal=International Statistical Review|volume=75|issue=2|year=2007|page=128|issn=0306-7734|doi=10.1111/j.1751-5823.2007.00009.x|s2cid=118011380 }}</ref>}} did not consider probability theory to be part of mathematics until the 20th century.<ref name="Chung1998"/><ref name="BenziBenzi2007"/><ref name="Doob1996">{{cite journal|last1=Doob|first1=Joseph L.|title=The Development of Rigor in Mathematical Probability (1900-1950)|journal=The American Mathematical Monthly|volume=103|issue=7|pages=586–595|year=1996|issn=0002-9890|doi=10.2307/2974673|jstor=2974673}}</ref><ref name="Cramer1976">{{cite journal|last1=Cramer|first1=Harald|title=Half a Century with Probability Theory: Some Personal Recollections|journal=The Annals of Probability|volume=4|issue=4|year=1976|pages=509–546|issn=0091-1798|doi=10.1214/aop/1176996025|doi-access=free}}</ref>

===Statistical mechanics===
In the physical sciences, scientists developed in the 19th century the discipline of [[statistical mechanics]], where physical systems, such as containers filled with gases, are regarded or treated mathematically as collections of many moving particles. Although there were attempts to incorporate randomness into statistical physics by some scientists, such as [[Rudolf Clausius]], most of the work had little or no randomness.<ref name="Truesdell1975page22">{{cite journal|last1=Truesdell|first1=C.|title=Early kinetic theories of gases|journal=Archive for History of Exact Sciences|volume=15|issue=1|year=1975|pages=22–23|issn=0003-9519|doi=10.1007/BF00327232|s2cid=189764116}}</ref><ref name="Brush1967page150">{{cite journal|last1=Brush|first1=Stephen G.|title=Foundations of statistical mechanics 1845?1915|journal=Archive for History of Exact Sciences|volume=4|issue=3|year=1967|pages=150–151|issn=0003-9519|doi=10.1007/BF00412958|s2cid=120059181}}</ref>
This changed in 1859 when [[James Clerk Maxwell]] contributed significantly to the field, more specifically, to the kinetic theory of gases, by presenting work where he modelled the gas particles as moving in random directions at random velocities.<ref name="Truesdell1975page31">{{cite journal|last1=Truesdell|first1=C.|title=Early kinetic theories of gases|journal=Archive for History of Exact Sciences|volume=15|issue=1|year=1975|pages=31–32|issn=0003-9519|doi=10.1007/BF00327232|s2cid=189764116}}</ref><ref name="Brush1958">{{cite journal|last1=Brush|first1=S.G.|title=The development of the kinetic theory of gases IV. Maxwell|journal=Annals of Science|volume=14|issue=4|year=1958|pages=243–255|issn=0003-3790|doi=10.1080/00033795800200147}}</ref> The kinetic theory of gases and statistical physics continued to be developed in the second half of the 19th century, with work done chiefly by Clausius, [[Ludwig Boltzmann]] and [[Josiah Gibbs]], which would later have an influence on [[Albert Einstein]]'s mathematical model for [[Brownian movement]].<ref name="Brush1968page15">{{cite journal|last1=Brush|first1=Stephen G.|title=A history of random processes|journal=Archive for History of Exact Sciences|volume=5|issue=1|year=1968|pages=15–16|issn=0003-9519|doi=10.1007/BF00328110|s2cid=117623580}}</ref>

===Measure theory and probability theory===
At the [[International Congress of Mathematicians]] in [[Paris]] in 1900, [[David Hilbert]] presented a list of [[Hilbert's problems|mathematical problems]], where his sixth problem asked for a mathematical treatment of physics and probability involving [[axiom]]s.<ref name="Bingham2000"/> Around the start of the 20th century, mathematicians developed measure theory, a branch of mathematics for studying integrals of mathematical functions, where two of the founders were French mathematicians, [[Henri Lebesgue]] and [[Émile Borel]]. In 1925, another French mathematician [[Paul Lévy (mathematician)|Paul Lévy]] published the first probability book that used ideas from measure theory.<ref name="Bingham2000"/>

In the 1920s, fundamental contributions to probability theory were made in the Soviet Union by mathematicians such as [[Sergei Bernstein]], [[Aleksandr Khinchin]],{{efn|The name Khinchin is also written in (or transliterated into) English as Khintchine.<ref name="Doob1934">{{cite journal|last1=Doob|first1=Joseph|title=Stochastic Processes and Statistics|journal=Proceedings of the National Academy of Sciences of the United States of America|volume=20|issue=6|year=1934|pages=376–379|doi=10.1073/pnas.20.6.376|pmid=16587907|pmc=1076423|bibcode=1934PNAS...20..376D|doi-access=free}}</ref>}} and [[Andrei Kolmogorov]].<ref name="Cramer1976"/> Kolmogorov published in 1929 his first attempt at presenting a mathematical foundation, based on measure theory, for probability theory.<ref name="KendallBatchelor1990page33">{{cite journal|last1=Kendall|first1=D. G.|last2=Batchelor|first2=G. K.|last3=Bingham|first3=N. H.|last4=Hayman|first4=W. K.|last5=Hyland|first5=J. M. E.|last6=Lorentz|first6=G. G.|last7=Moffatt|first7=H. K.|last8=Parry|first8=W.|last9=Razborov|first9=A. A.|last10=Robinson|first10=C. A.|last11=Whittle|first11=P.|title=Andrei Nikolaevich Kolmogorov (1903–1987)|journal=Bulletin of the London Mathematical Society|volume=22|issue=1|year=1990|page=33|issn=0024-6093|doi=10.1112/blms/22.1.31}}</ref> In the early 1930s, Khinchin and Kolmogorov set up probability seminars, which were attended by researchers such as [[Eugene Slutsky]] and [[Nikolai Smirnov (mathematician)|Nikolai Smirnov]],<ref name="Vere-Jones2006page1">{{cite book|last1=Vere-Jones|first1=David|title=Encyclopedia of Statistical Sciences|chapter=Khinchin, Aleksandr Yakovlevich|page=1|year=2006|doi=10.1002/0471667196.ess6027.pub2|isbn=978-0471667193}}</ref> and Khinchin gave the first mathematical definition of a stochastic process as a set of random variables indexed by the real line.<ref name="Doob1934"/><ref name="Vere-Jones2006page4">{{cite book|last1=Vere-Jones|first1=David|title=Encyclopedia of Statistical Sciences|chapter=Khinchin, Aleksandr Yakovlevich|page=4|year=2006|doi=10.1002/0471667196.ess6027.pub2|isbn=978-0471667193}}</ref>{{efn|Doob, when citing Khinchin, uses the term 'chance variable', which used to be an alternative term for 'random variable'.<ref name="Snell2005">{{cite journal|last1=Snell|first1=J. Laurie|title=Obituary: Joseph Leonard Doob|journal=Journal of Applied Probability|volume=42|issue=1|year=2005|page=251|issn=0021-9002|doi=10.1239/jap/1110381384|doi-access=free}}</ref> }}

===Birth of modern probability theory===
In 1933, Andrei Kolmogorov published in German, his book on the foundations of probability theory titled ''Grundbegriffe der Wahrscheinlichkeitsrechnung'',{{efn|Later translated into English and published in 1950 as Foundations of the Theory of Probability<ref name="Bingham2000"/>}} where Kolmogorov used measure theory to develop an axiomatic framework for probability theory. The publication of this book is now widely considered to be the birth of modern probability theory, when the theories of probability and stochastic processes became parts of mathematics.<ref name="Bingham2000"/><ref name="Cramer1976"/>

After the publication of Kolmogorov's book, further fundamental work on probability theory and stochastic processes was done by Khinchin and Kolmogorov as well as other mathematicians such as [[Joseph Doob]], [[William Feller]], [[Maurice Fréchet]], [[Paul Lévy (mathematician)|Paul Lévy]], [[Wolfgang Doeblin]], and [[Harald Cramér]].<ref name="Bingham2000"/><ref name="Cramer1976"/>
Decades later, Cramér referred to the 1930s as the "heroic period of mathematical probability theory".<ref name="Cramer1976"/> [[World War II]] greatly interrupted the development of probability theory, causing, for example, the migration of Feller from [[Sweden]] to the [[United States|United States of America]]<ref name="Cramer1976"/> and the death of Doeblin, considered now a pioneer in stochastic processes.<ref name="Lindvall1991">{{cite journal|last1=Lindvall|first1=Torgny|title=W. Doeblin, 1915-1940|journal=The Annals of Probability|volume=19|issue=3|year=1991|pages=929–934|issn=0091-1798|doi=10.1214/aop/1176990329|doi-access=free}}</ref>

[[File:Joseph Doob.jpg|thumb|right|Mathematician [[Joseph Doob]] did early work on the theory of stochastic processes, making fundamental contributions, particularly in the theory of martingales.<ref name="Getoor2009"/><ref name="Snell2005"/> His book ''Stochastic Processes'' is considered highly influential in the field of probability theory.<ref name="Bingham2005"/> ]]

===Stochastic processes after World War II===
After World War II, the study of probability theory and stochastic processes gained more attention from mathematicians, with significant contributions made in many areas of probability and mathematics as well as the creation of new areas.<ref name="Cramer1976"/><ref name="Meyer2009">{{cite journal|last1=Meyer|first1=Paul-André|title=Stochastic Processes from 1950 to the Present|journal=Electronic Journal for History of Probability and Statistics|volume=5|issue=1|year=2009|pages=1–42}}</ref> Starting in the 1940s, [[Kiyosi Itô]] published papers developing the field of [[stochastic calculus]], which involves stochastic [[integrals]] and stochastic [[differential equations]] based on the Wiener or Brownian motion process.<ref name="Ito1998Prize">{{cite journal|title=Kiyosi Itô receives Kyoto Prize|journal=Notices of the AMS|volume=45|issue=8|year=1998|pages=981–982}}</ref>

Also starting in the 1940s, connections were made between stochastic processes, particularly martingales, and the mathematical field of [[potential theory]], with early ideas by [[Shizuo Kakutani]] and then later work by Joseph Doob.<ref name="Meyer2009"/> Further work, considered pioneering, was done by [[Gilbert Hunt]] in the 1950s, connecting Markov processes and potential theory, which had a significant effect on the theory of Lévy processes and led to more interest in studying Markov processes with methods developed by Itô.<ref name="JarrowProtter2004"/><ref name="Bertoin1998pageVIIIandIX">{{cite book|author=Jean Bertoin|title=Lévy Processes|url=https://books.google.com/books?id=ftcsQgMp5cUC&pg=PR8|year=1998|publisher=Cambridge University Press|isbn=978-0-521-64632-1|page=viii and ix}}</ref><ref name="Steele2012page176">{{cite book|author=J. Michael Steele|title=Stochastic Calculus and Financial Applications|url=https://books.google.com/books?id=fsgkBAAAQBAJ&pg=PR4|year=2012|publisher=Springer Science & Business Media|isbn=978-1-4684-9305-4|page=176}}</ref>

In 1953, Doob published his book ''Stochastic processes'', which had a strong influence on the theory of stochastic processes and stressed the importance of measure theory in probability.<ref name="Meyer2009"/>
<ref name="Bingham2005">{{cite journal|last1=Bingham|first1=N. H.|title=Doob: a half-century on|journal=Journal of Applied Probability|volume=42|issue=1|year=2005|pages=257–266|issn=0021-9002|doi=10.1239/jap/1110381385|doi-access=free}}</ref> Doob also chiefly developed the theory of martingales, with later substantial contributions by [[Paul-André Meyer]]. Earlier work had been carried out by [[Sergei Bernstein]], [[Paul Lévy (mathematician)|Paul Lévy]] and [[Jean Ville]], the latter adopting the term martingale for the stochastic process.<ref name="HallHeyde2014page1">{{cite book|author1=P. Hall|author2=C. C. Heyde|title=Martingale Limit Theory and Its Application|url=https://books.google.com/books?id=gqriBQAAQBAJ&pg=PR10|year=2014|publisher=Elsevier Science|isbn=978-1-4832-6322-9|pages=1, 2}}</ref><ref name="Dynkin1989">{{cite journal|last1=Dynkin|first1=E. B.|title=Kolmogorov and the Theory of Markov Processes|journal=The Annals of Probability|volume=17|issue=3|year=1989|pages=822–832|issn=0091-1798|doi=10.1214/aop/1176991248|doi-access=free}}</ref> Methods from the theory of martingales became popular for solving various probability problems. Techniques and theory were developed to study Markov processes and then applied to martingales. Conversely, methods from the theory of martingales were established to treat Markov processes.<ref name="Meyer2009"/>

Other fields of probability were developed and used to study stochastic processes, with one main approach being the theory of large deviations.<ref name="Meyer2009"/> The theory has many applications in statistical physics, among other fields, and has core ideas going back to at least the 1930s. Later in the 1960s and 1970s, fundamental work was done by Alexander Wentzell in the Soviet Union and [[Monroe D. Donsker]] and [[Srinivasa Varadhan]] in the United States of America,<ref name="Ellis1995page98">{{cite journal|last1=Ellis|first1=Richard S.|title=An overview of the theory of large deviations and applications to statistical mechanics|journal=Scandinavian Actuarial Journal|volume=1995|issue=1|year=1995|page=98|issn=0346-1238|doi=10.1080/03461238.1995.10413952}}</ref> which would later result in Varadhan winning the 2007 Abel Prize.<ref name="RaussenSkau2008">{{cite journal|last1=Raussen|first1=Martin|last2=Skau|first2=Christian|title=Interview with Srinivasa Varadhan|journal=Notices of the AMS|volume=55|issue=2|year=2008|pages=238–246}}</ref> In the 1990s and 2000s the theories of [[Schramm–Loewner evolution]]<ref name="HenkelKarevski2012page113">{{cite book|author1=Malte Henkel|author2=Dragi Karevski|title=Conformal Invariance: an Introduction to Loops, Interfaces and Stochastic Loewner Evolution|url=https://books.google.com/books?id=fnCQWd0GEZ8C&pg=PA113|year=2012|publisher=Springer Science & Business Media|isbn=978-3-642-27933-1|page=113}}</ref> and [[rough paths]]<ref name="FrizVictoir2010page571">{{cite book|author1=Peter K. Friz|author2=Nicolas B. Victoir|author1-link=Peter Friz|title=Multidimensional Stochastic Processes as Rough Paths: Theory and Applications|url=https://books.google.com/books?id=CVgwLatxfGsC|year=2010|publisher=Cambridge University Press|isbn=978-1-139-48721-4|page=571}}</ref> were introduced and developed to study stochastic processes and other mathematical objects in probability theory, which respectively resulted in [[Fields Medal]]s being awarded to [[Wendelin Werner]]<ref name="Werner2004Fields">{{cite journal|title=2006 Fields Medals Awarded|journal=Notices of the AMS|volume=53|issue=9|year=2015|pages=1041–1044}}</ref> in 2008 and to [[Martin Hairer]] in 2014.<ref name="Hairer2004Fields">{{cite journal|last1=Quastel|first1=Jeremy|title=The Work of the 2014 Fields Medalists|journal=Notices of the AMS|volume=62|issue=11|year=2015|pages=1341–1344}}</ref>

The theory of stochastic processes still continues to be a focus of research, with yearly international conferences on the topic of stochastic processes.<ref name="BlathImkeller2011"/><ref name="Applebaum2004page1336"/>

===Discoveries of specific stochastic processes===
Although Khinchin gave mathematical definitions of stochastic processes in the 1930s,<ref name="Doob1934"/><ref name="Vere-Jones2006page4"/> specific stochastic processes had already been discovered in different settings, such as the Brownian motion process and the Poisson process.<ref name="JarrowProtter2004"/><ref name="GuttorpThorarinsdottir2012"/> Some families of stochastic processes such as point processes or renewal processes have long and complex histories, stretching back centuries.<ref name="DaleyVere-Jones2006chap1">{{cite book|author1=D.J. Daley|author2=D. Vere-Jones|title=An Introduction to the Theory of Point Processes: Volume I: Elementary Theory and Methods|url=https://books.google.com/books?id=6Sv4BwAAQBAJ|year=2006|publisher=Springer Science & Business Media|isbn=978-0-387-21564-8|pages=1–4}}</ref>

====Bernoulli process====
The Bernoulli process, which can serve as a mathematical model for flipping a biased coin, is possibly the first stochastic process to have been studied.<ref name="Florescu2014page301"/> The process is a sequence of independent Bernoulli trials,<ref name="BertsekasTsitsiklis2002page273"/> which are named after [[Jacob Bernoulli]] who used them to study games of chance, including probability problems proposed and studied earlier by Christiaan Huygens.<ref name="Hald2005page226">{{cite book|author=Anders Hald|title=A History of Probability and Statistics and Their Applications before 1750|url=https://books.google.com/books?id=pOQy6-qnVx8C|year=2005|publisher=John Wiley & Sons|isbn=978-0-471-72517-6|page=226}}</ref> Bernoulli's work, including the Bernoulli process, were published in his book ''Ars Conjectandi'' in 1713.<ref name="Lebowitz1984">{{cite book|author=Joel Louis Lebowitz|title=Nonequilibrium phenomena II: from stochastics to hydrodynamics|url=https://books.google.com/books?id=E8IRAQAAIAAJ|year=1984|publisher=North-Holland Pub.|isbn=978-0-444-86806-0|pages=8–10}}</ref>

====Random walks====
In 1905, [[Karl Pearson]] coined the term ''random walk'' while posing a problem describing a random walk on the plane, which was motivated by an application in biology, but such problems involving random walks had already been studied in other fields. Certain gambling problems that were studied centuries earlier can be considered as problems involving random walks.<ref name="Weiss2006page1"/><ref name="Lebowitz1984"/> For example, the problem known as the ''Gambler's ruin'' is based on a simple random walk,<ref name="KarlinTaylor2012page49"/><ref name="Florescu2014page374">{{cite book|author=Ionut Florescu|title=Probability and Stochastic Processes|url=https://books.google.com/books?id=Z5xEBQAAQBAJ&pg=PR22|year=2014|publisher=John Wiley & Sons|isbn=978-1-118-59320-2|pages=374}}</ref> and is an example of a random walk with absorbing barriers.<ref name="Seneta2006page1"/><ref name="Ibe2013page5">{{cite book|author=Oliver C. Ibe|title=Elements of Random Walk and Diffusion Processes|url=https://books.google.com/books?id=DUqaAAAAQBAJ&pg=PT10|year=2013|publisher=John Wiley & Sons|isbn=978-1-118-61793-9|page=5}}</ref> Pascal, Fermat and Huyens all gave numerical solutions to this problem without detailing their methods,<ref name="Hald2005page63">{{cite book|author=Anders Hald|title=A History of Probability and Statistics and Their Applications before 1750|url=https://books.google.com/books?id=pOQy6-qnVx8C|year=2005|publisher=John Wiley & Sons|isbn=978-0-471-72517-6|page=63}}</ref> and then more detailed solutions were presented by Jakob Bernoulli and [[Abraham de Moivre]].<ref name="Hald2005page202">{{cite book|author=Anders Hald|title=A History of Probability and Statistics and Their Applications before 1750|url=https://books.google.com/books?id=pOQy6-qnVx8C|year=2005|publisher=John Wiley & Sons|isbn=978-0-471-72517-6|page=202}}</ref>

For random walks in <math>n</math>-dimensional integer [[Lattice (group)|lattices]], [[George Pólya]] published, in 1919 and 1921, work where he studied the probability of a symmetric random walk returning to a previous position in the lattice. Pólya showed that a symmetric random walk, which has an equal probability to advance in any direction in the lattice, will return to a previous position in the lattice an infinite number of times with probability one in one and two dimensions, but with probability zero in three or higher dimensions.<ref name="Florescu2014page385">{{cite book|author=Ionut Florescu|title=Probability and Stochastic Processes|url=https://books.google.com/books?id=Z5xEBQAAQBAJ&pg=PR22|year=2014|publisher=John Wiley & Sons|isbn=978-1-118-59320-2|pages=385}}</ref><ref name="Hughes1995page111">{{cite book|author=Barry D. Hughes|title=Random Walks and Random Environments: Random walks|url=https://books.google.com/books?id=QhOen_t0LeQC|year=1995|publisher=Clarendon Press|isbn=978-0-19-853788-5|page=111}}</ref>

====Wiener process====
The [[Wiener process]] or Brownian motion process has its origins in different fields including statistics, finance and physics.<ref name="JarrowProtter2004"/> In 1880, Danish astronomer  [[Thorvald Thiele]] wrote a paper on the method of least squares, where he used the process to study the errors of a model in time-series analysis.<ref name="Thiele1880">{{cite journal|last1=Thiele|first1=Thorwald N.|title=Om Anvendelse af mindste Kvadraterbs Methode i nogle Tilfælde, hvoren Komplikation af visse Slags uensartede tilfældige Fejlkilder giver Fejleneen "systematisk" Karakter|journal=Kongelige Danske Videnskabernes Selskabs Skrifter |volume=Series 5|issue=12|year=1880|pages=381–408|url=https://biodiversitylibrary.org/page/43213604}}</ref><ref name="Hald1981page1and18">{{cite journal|last1=Hald|first1=Anders |title=T. N. Thiele's Contributions to Statistics|journal=International Statistical Review / Revue Internationale de Statistique|volume=49|issue=1|year=1981|pages=1–20|issn=0306-7734|doi=10.2307/1403034|jstor=1403034}}</ref><ref name="Lauritzen1981page319">{{cite journal|last1=Lauritzen|first1=Steffen L.|title=Time Series Analysis in 1880: A Discussion of Contributions Made by T.N. Thiele|journal=International Statistical Review / Revue Internationale de Statistique|volume=49|issue=3|year=1981|pages=319–320|issn=0306-7734|doi=10.2307/1402616|jstor=1402616}}</ref> The work is now considered as an early discovery of the statistical method known as [[Kalman filtering]], but the work was largely overlooked. It is thought that the ideas in Thiele's paper were too advanced to have been understood by the broader mathematical and statistical community at the time.<ref name="Lauritzen1981page319"/>

[[File:Wiener Zurich1932.tif|thumb|200px|[[Norbert Wiener]] gave the first mathematical proof of the existence of the Wiener process. This mathematical object had appeared previously in the work of [[Thorvald Thiele]], [[Louis Bachelier]], and [[Albert Einstein]].<ref name="JarrowProtter2004"/>]]

The French mathematician [[Louis Bachelier]] used a Wiener process in his 1900 thesis<ref name=Bachelier1900a>{{cite journal |last=Bachelier |first=Luis |year=1900 |title=Théorie de la spéculation |journal=[[Ann. Sci. Éc. Norm. Supér.]] |volume=Serie 3;17 |pages=21–89 |url=http://archive.numdam.org/article/ASENS_1900_3_17__21_0.pdf |archive-url=https://web.archive.org/web/20110605013545/http://archive.numdam.org/article/ASENS_1900_3_17__21_0.pdf |archive-date=2011-06-05 |url-status=live |doi=10.24033/asens.476 |doi-access=free }}</ref><ref name=Bachelier1900b>{{cite journal |last=Bachelier |first=Luis |year=1900 |title=The Theory of Speculation |journal=Ann. Sci. Éc. Norm. Supér. |volume=Serie 3;17 |pages=21–89 (Engl. translation by David R. May, 2011) |url=https://drive.google.com/file/d/0B5LLDy7-d3SKNGI0M2E0NGItYzFlMS00NGU2LWE2ZDAtODc3MDY3MzdiNmY0/view |doi=10.24033/asens.476 |doi-access=free }}</ref> in order to model price changes on the [[Paris Bourse]], a [[stock exchange]],<ref name="CourtaultKabanov2000">{{cite journal|last1=Courtault|first1=Jean-Michel|last2=Kabanov|first2=Yuri|last3=Bru|first3=Bernard|last4=Crepel|first4=Pierre|last5=Lebon|first5=Isabelle|last6=Le Marchand|first6=Arnaud|title=Louis Bachelier on the Centenary of Theorie de la Speculation|journal=Mathematical Finance|volume=10|issue=3|year=2000|pages=339–353|issn=0960-1627|doi=10.1111/1467-9965.00098|s2cid=14422885 |url=https://halshs.archives-ouvertes.fr/halshs-00447592/file/BACHEL2.PDF |archive-url=https://web.archive.org/web/20180721214136/https://halshs.archives-ouvertes.fr/halshs-00447592/file/BACHEL2.PDF |archive-date=2018-07-21 |url-status=live}}</ref> without knowing the work of Thiele.<ref name="JarrowProtter2004"/> It has been speculated that Bachelier drew ideas from the random walk model of [[Jules Regnault]], but Bachelier did not cite him,<ref name="Jovanovic2012">{{cite journal|last1=Jovanovic|first1=Franck|title=Bachelier: Not the forgotten forerunner he has been depicted as. An analysis of the dissemination of Louis Bachelier's work in economics|journal=The European Journal of the History of Economic Thought|volume=19|issue=3|year=2012|pages=431–451|issn=0967-2567|doi=10.1080/09672567.2010.540343|s2cid=154003579|url=http://r-libre.teluq.ca/1168/1/dissemination%20of%20Louis%20Bachelier_EJHET_R2.pdf |archive-url=https://web.archive.org/web/20180721111017/http://r-libre.teluq.ca/1168/1/dissemination%20of%20Louis%20Bachelier_EJHET_R2.pdf |archive-date=2018-07-21 |url-status=live}}</ref> and Bachelier's thesis is now considered pioneering in the field of financial mathematics.<ref name="CourtaultKabanov2000"/><ref name="Jovanovic2012"/>

It is commonly thought that Bachelier's work gained little attention and was forgotten for decades until it was rediscovered in the 1950s by the [[Leonard Savage]], and then become more popular after Bachelier's thesis was translated into English in 1964. But the work was never forgotten in the mathematical community, as Bachelier published a book in 1912 detailing his ideas,<ref name="Jovanovic2012"/> which was cited by mathematicians including Doob, Feller<ref name="Jovanovic2012"/> and Kolmogorov.<ref name="JarrowProtter2004"/> The book continued to be cited, but then starting in the 1960s, the original thesis by Bachelier began to be cited more than his book when economists started citing Bachelier's work.<ref name="Jovanovic2012"/>

In 1905, [[Albert Einstein]] published a paper where he studied the physical observation of Brownian motion or movement to explain the seemingly random movements of particles in liquids by using ideas from the [[kinetic theory of gases]]. Einstein derived a [[differential equation]], known as a [[diffusion equation]], for describing the probability of finding a particle in a certain region of space. Shortly after Einstein's first paper on Brownian movement, [[Marian Smoluchowski]] published work where he cited Einstein, but wrote that he had independently derived the equivalent results by using a different method.<ref name="Brush1968page25">{{cite journal|last1=Brush|first1=Stephen G.|title=A history of random processes|journal=Archive for History of Exact Sciences|volume=5|issue=1|year=1968|page=25|issn=0003-9519|doi=10.1007/BF00328110|s2cid=117623580}}</ref>

Einstein's work, as well as experimental results obtained by [[Jean Perrin]], later inspired Norbert Wiener in the 1920s<ref name="Brush1968page30">{{cite journal|last1=Brush|first1=Stephen G.|title=A history of random processes|journal=Archive for History of Exact Sciences|volume=5|issue=1|year=1968|pages=1–36|issn=0003-9519|doi=10.1007/BF00328110|s2cid=117623580}}</ref> to use a type of measure theory, developed by [[Percy Daniell]], and Fourier analysis to prove the existence of the Wiener process as a mathematical object.<ref name="JarrowProtter2004"/>

====Poisson process====
The Poisson process is named after [[Siméon Poisson]], due to its definition involving the [[Poisson distribution]], but Poisson never studied the process.<ref name="Stirzaker2000"/><ref name="DaleyVere-Jones2006page8">{{cite book|author1=D.J. Daley|author2=D. Vere-Jones|title=An Introduction to the Theory of Point Processes: Volume I: Elementary Theory and Methods|url=https://books.google.com/books?id=6Sv4BwAAQBAJ|year=2006|publisher=Springer Science & Business Media|isbn=978-0-387-21564-8|pages=8–9}}</ref> There are a number of claims for early uses or discoveries of the Poisson
process.<ref name="Stirzaker2000"/><ref name="GuttorpThorarinsdottir2012"/>
At the beginning of the 20th century, the Poisson process would arise independently in different situations.<ref name="Stirzaker2000"/><ref name="GuttorpThorarinsdottir2012"/>
In Sweden 1903, [[Filip Lundberg]] published a [[thesis]] containing work, now considered fundamental and pioneering, where he proposed to model insurance claims with a homogeneous Poisson process.<ref name="EmbrechtsFrey2001page367">{{cite book|last1=Embrechts|first1=Paul|title=Stochastic Processes: Theory and Methods|last2=Frey|first2=Rüdiger|last3=Furrer|first3=Hansjörg|chapter=Stochastic processes in insurance and finance|volume=19|year=2001|page=367|issn=0169-7161|doi=10.1016/S0169-7161(01)19014-0|series=Handbook of Statistics|isbn=978-0444500144}}</ref><ref name="Cramér1969">{{cite journal|last1=Cramér|first1=Harald|title=Historical review of Filip Lundberg's works on risk theory|journal=Scandinavian Actuarial Journal|volume=1969|issue=sup3|year=1969|pages=6–12|issn=0346-1238|doi=10.1080/03461238.1969.10404602}}</ref>

Another discovery occurred in [[Denmark]] in 1909 when [[A.K. Erlang]] derived the Poisson distribution when developing a mathematical model for the number of incoming phone calls in a finite time interval. Erlang was not at the time aware of Poisson's earlier work and assumed that the number phone calls arriving in each interval of time were independent to each other. He then found the limiting case, which is effectively recasting the Poisson distribution as a limit of the binomial distribution.<ref name="Stirzaker2000"/>

In 1910, [[Ernest Rutherford]] and [[Hans Geiger]] published experimental results on counting alpha particles. Motivated by their work, [[Harry Bateman]] studied the counting problem and derived Poisson probabilities as a solution to a family of differential equations, resulting in the independent discovery of the Poisson process.<ref name="Stirzaker2000"/> After this time there were many studies and applications of the Poisson process, but its early history is complicated, which has been explained by the various applications of the process in numerous fields by biologists, ecologists, engineers and various physical scientists.<ref name="Stirzaker2000"/>

====Markov processes====
Markov processes and Markov chains are named after [[Andrey Markov]] who studied Markov chains in the early 20th century. Markov was interested in studying an extension of independent random sequences. In his first paper on Markov chains, published in 1906, Markov showed that under certain conditions the average outcomes of the Markov chain would converge to a fixed vector of values, so proving a [[weak law of large numbers]] without the independence assumption,<ref name="GrinsteadSnell1997page464">{{cite book|author1=Charles Miller Grinstead|author2=James Laurie Snell|title=Introduction to Probability|url=https://archive.org/details/flooved3489|year=1997|publisher=American Mathematical Soc.|isbn=978-0-8218-0749-1|pages=[https://archive.org/details/flooved3489/page/n473 464]–466}}</ref><ref name="Bremaud2013pageIX">{{cite book|author=Pierre Bremaud|title=Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues|url=https://books.google.com/books?id=jrPVBwAAQBAJ|year=2013|publisher=Springer Science & Business Media|isbn=978-1-4757-3124-8|page=ix}}</ref><ref name="Hayes2013">{{cite journal|last1=Hayes|first1=Brian|title=First links in the Markov chain|journal=American Scientist|volume=101|issue=2|year=2013|pages=92–96|doi=10.1511/2013.101.92}}</ref> which had been commonly regarded as a requirement for such mathematical laws to hold.<ref name="Hayes2013"/> Markov later used Markov chains to study the distribution of vowels in [[Eugene Onegin]], written by [[Alexander Pushkin]], and proved a [[central limit theorem]] for such chains.

In 1912, Poincaré studied Markov chains on [[finite group]]s with an aim to study card shuffling. Other early uses of Markov chains include a diffusion model, introduced by [[Paul Ehrenfest|Paul]] and [[Tatyana Ehrenfest]] in 1907, and a branching process, introduced by [[Francis Galton]] and [[Henry William Watson]] in 1873, preceding the work of Markov.<ref name="GrinsteadSnell1997page464"/><ref name="Bremaud2013pageIX"/> After the work of Galton and Watson, it was later revealed that their branching process had been independently discovered and studied around three decades earlier by [[Irénée-Jules Bienaymé]].<ref name="Seneta1998">{{cite journal|last1=Seneta|first1=E.|title=I.J. Bienaymé [1796-1878]: Criticality, Inequality, and Internationalization|journal=International Statistical Review / Revue Internationale de Statistique|volume=66|issue=3|year=1998|pages=291–292|issn=0306-7734|doi=10.2307/1403518|jstor=1403518}}</ref> Starting in 1928, [[Maurice Fréchet]] became interested in Markov chains, eventually resulting in him publishing in 1938 a detailed study on Markov chains.<ref name="GrinsteadSnell1997page464"/><ref name="BruHertz2001">{{cite book|last1=Bru|first1=B.|title=Statisticians of the Centuries|last2=Hertz|first2=S.|chapter=Maurice Fréchet|year=2001|pages=331–334|doi=10.1007/978-1-4613-0179-0_71|isbn=978-0-387-95283-3}}</ref>

[[Andrei Kolmogorov]] developed in a 1931 paper a large part of the early theory of continuous-time Markov processes.<ref name="Cramer1976"/><ref name="KendallBatchelor1990page33"/> Kolmogorov was partly inspired by Louis Bachelier's 1900 work on fluctuations in the stock market as well as [[Norbert Wiener]]'s work on Einstein's model of Brownian movement.<ref name="KendallBatchelor1990page33"/><ref name="BarbutLocker2016page5">{{cite book|author1=Marc Barbut|author2=Bernard Locker|author3=Laurent Mazliak|title=Paul Lévy and Maurice Fréchet: 50 Years of Correspondence in 107 Letters|url=https://books.google.com/books?id=lSz_vQAACAAJ|date= 2016|publisher=Springer London|isbn=978-1-4471-7262-8|page=5}}</ref> He introduced and studied a particular set of Markov processes known as diffusion processes, where he derived a set of differential equations describing the processes.<ref name="KendallBatchelor1990page33"/><ref name="Skorokhod2005page146">{{cite book|author=Valeriy Skorokhod|title=Basic Principles and Applications of Probability Theory|url=https://books.google.com/books?id=dQkYMjRK3fYC|year=2005|publisher=Springer Science & Business Media|isbn=978-3-540-26312-8|page=146}}</ref> Independent of Kolmogorov's work, [[Sydney Chapman (mathematician)|Sydney Chapman]] derived in a 1928 paper an equation, now called the [[Chapman–Kolmogorov equation]], in a less mathematically rigorous way than Kolmogorov, while studying Brownian movement.<ref name="Bernstein2005">{{cite journal|last1=Bernstein|first1=Jeremy|title=Bachelier|journal=American Journal of Physics|volume=73|issue=5|year=2005|pages=398–396|issn=0002-9505|doi=10.1119/1.1848117|bibcode=2005AmJPh..73..395B}}</ref> The differential equations are now called the Kolmogorov equations<ref name="Anderson2012pageVII">{{cite book|author=William J. Anderson|title=Continuous-Time Markov Chains: An Applications-Oriented Approach|url=https://books.google.com/books?id=YpHfBwAAQBAJ&pg=PR8|year=2012|publisher=Springer Science & Business Media|isbn=978-1-4612-3038-0|page=vii}}</ref> or the Kolmogorov–Chapman equations.<ref name="KendallBatchelor1990page57">{{cite journal|last1=Kendall|first1=D. G.|last2=Batchelor|first2=G. K.|last3=Bingham|first3=N. H.|last4=Hayman|first4=W. K.|last5=Hyland|first5=J. M. E.|last6=Lorentz|first6=G. G.|last7=Moffatt|first7=H. K.|last8=Parry|first8=W.|last9=Razborov|first9=A. A.|last10=Robinson|first10=C. A.|last11=Whittle|first11=P.|title=Andrei Nikolaevich Kolmogorov (1903–1987)|journal=Bulletin of the London Mathematical Society|volume=22|issue=1|year=1990|page=57|issn=0024-6093|doi=10.1112/blms/22.1.31}}</ref> Other mathematicians who contributed significantly to the foundations of Markov processes include William Feller, starting in the 1930s, and then later Eugene Dynkin, starting in the 1950s.<ref name="Cramer1976"/>

====Lévy processes====
Lévy processes such as the Wiener process and the Poisson process (on the real line) are named after Paul Lévy who started studying them in the 1930s,<ref name="Applebaum2004page1336"/> but they have connections to [[infinitely divisible distribution]]s going back to the 1920s.<ref name="Bertoin1998pageVIII"/> In a 1932 paper, Kolmogorov derived a [[Characteristic function (probability theory)|characteristic function]] for random variables associated with Lévy processes. This result was later derived under more general conditions by Lévy in 1934, and then Khinchin independently gave an alternative form for this characteristic function in 1937.<ref name="Cramer1976"/><ref name="ApplebaumBook2004page67">{{cite book|author=David Applebaum|title=Lévy Processes and Stochastic Calculus|url=https://books.google.com/books?id=q7eDUjdJxIkC|year=2004|publisher=Cambridge University Press|isbn=978-0-521-83263-2|page=67}}</ref> In addition to Lévy, Khinchin and Kolomogrov, early fundamental contributions to the theory of Lévy processes were made by [[Bruno de Finetti]] and [[Kiyosi Itô]].<ref name="Bertoin1998pageVIII"/>