Editing Stochastic process (section)

==Further examples==

===Markov processes and chains===
{{Main|Markov chain}}
Markov processes are stochastic processes, traditionally in [[Discrete time and continuous time|discrete or continuous time]], that have the Markov property, which means the next value of the Markov process depends on the current value, but it is conditionally independent of the previous values of the stochastic process. In other words, the behavior of the process in the future is stochastically independent of its behavior in the past, given the current state of the process.<ref name="Serfozo2009page2">{{cite book|author=Richard Serfozo|title=Basics of Applied Stochastic Processes|url=https://books.google.com/books?id=JBBRiuxTN0QC|year=2009|publisher=Springer Science & Business Media|isbn=978-3-540-89332-5|page=2}}</ref><ref name="Rozanov2012page58">{{cite book|author=Y.A. Rozanov|title=Markov Random Fields|url=https://books.google.com/books?id=wGUECAAAQBAJ|year=2012|publisher=Springer Science & Business Media|isbn=978-1-4613-8190-7|page=58}}</ref>

The Brownian motion process and the Poisson process (in one dimension) are both examples of Markov processes<ref name="Ross1996page235and358">{{cite book|author=Sheldon M. Ross|title=Stochastic processes|url=https://books.google.com/books?id=ImUPAQAAMAAJ|year=1996|publisher=Wiley|isbn=978-0-471-12062-9|pages=235, 358}}</ref> in continuous time, while [[random walk]]s on the integers and the [[gambler's ruin]] problem are examples of Markov processes in discrete time.<ref name="Florescu2014page373">{{cite book|author=Ionut Florescu|title=Probability and Stochastic Processes|url=https://books.google.com/books?id=Z5xEBQAAQBAJ&pg=PR22|year=2014|publisher=John Wiley & Sons|isbn=978-1-118-59320-2|pages=373, 374}}</ref><ref name="KarlinTaylor2012page49">{{cite book|author1=Samuel Karlin|author2=Howard E. Taylor|title=A First Course in Stochastic Processes|url=https://books.google.com/books?id=dSDxjX9nmmMC|year=2012|publisher=Academic Press|isbn=978-0-08-057041-9|page=49}}</ref>

A Markov chain is a type of Markov process that has either discrete [[state space]] or discrete index set (often representing time), but the precise definition of a Markov chain varies.<ref name="Asmussen2003page7">{{cite book|url=https://books.google.com/books?id=BeYaTxesKy0C|title=Applied Probability and Queues|year=2003|publisher=Springer Science & Business Media|isbn=978-0-387-00211-8|page=7|author=Søren Asmussen}}</ref> For example, it is common to define a Markov chain as a Markov process in either [[Continuous and discrete variables|discrete or continuous time]] with a countable state space (thus regardless of the nature of time),<ref name="Parzen1999page188">{{cite book|url=https://books.google.com/books?id=0mB2CQAAQBAJ|title=Stochastic Processes|year=2015|publisher=Courier Dover Publications|isbn=978-0-486-79688-8|page=188|author=Emanuel Parzen}}</ref><ref name="KarlinTaylor2012page29">{{cite book|url=https://books.google.com/books?id=dSDxjX9nmmMC|title=A First Course in Stochastic Processes|year=2012|publisher=Academic Press|isbn=978-0-08-057041-9|pages=29, 30|author1=Samuel Karlin|author2=Howard E. Taylor}}</ref><ref name="Lamperti1977chap6">{{cite book|url=https://books.google.com/books?id=Pd4cvgAACAAJ|title=Stochastic processes: a survey of the mathematical theory|publisher=Springer-Verlag|year=1977|isbn=978-3-540-90275-1|pages=106–121|author=John Lamperti}}</ref><ref name="Ross1996page174and231">{{cite book|url=https://books.google.com/books?id=ImUPAQAAMAAJ|title=Stochastic processes|publisher=Wiley|year=1996|isbn=978-0-471-12062-9|pages=174, 231|author=Sheldon M. Ross}}</ref> but it has been also common to define a Markov chain as having discrete time in either countable or continuous state space (thus regardless of the state space).<ref name="Asmussen2003page7" /> It has been argued that the first definition of a Markov chain, where it has discrete time, now tends to be used, despite the second definition having been used by researchers like [[Joseph Doob]] and [[Kai Lai Chung]].<ref name="MeynTweedie2009">{{cite book|author1=Sean Meyn|author2=Richard L. Tweedie|title=Markov Chains and Stochastic Stability|url=https://books.google.com/books?id=Md7RnYEPkJwC|year=2009|publisher=Cambridge University Press|isbn=978-0-521-73182-9|page=19}}</ref>

Markov processes form an important class of stochastic processes and have applications in many areas.<ref name="LatoucheRamaswami1999"/><ref name="KarlinTaylor2012page47">{{cite book|author1=Samuel Karlin|author2=Howard E. Taylor|title=A First Course in Stochastic Processes|url=https://books.google.com/books?id=dSDxjX9nmmMC|year=2012|publisher=Academic Press|isbn=978-0-08-057041-9|page=47}}</ref> For example, they are the basis for a general stochastic simulation method known as [[Markov chain Monte Carlo]], which is used for simulating random objects with specific probability distributions, and has found application in [[Bayesian statistics]].<ref name="RubinsteinKroese2011page225">{{cite book|author1=Reuven Y. Rubinstein|author2=Dirk P. Kroese|title=Simulation and the Monte Carlo Method|url=https://books.google.com/books?id=yWcvT80gQK4C|year=2011|publisher=John Wiley & Sons|isbn=978-1-118-21052-9|page=225}}</ref><ref name="GamermanLopes2006">{{cite book|author1=Dani Gamerman|author2=Hedibert F. Lopes|title=Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference, Second Edition|url=https://books.google.com/books?id=yPvECi_L3bwC|year=2006|publisher=CRC Press|isbn=978-1-58488-587-0}}</ref>

The concept of the Markov property was originally for stochastic processes in continuous and discrete time, but the property has been adapted for other index sets such as <math>n</math>-dimensional Euclidean space, which results in collections of random variables known as Markov random fields.<ref name="Rozanov2012page61">{{cite book|author=Y.A. Rozanov|title=Markov Random Fields|url=https://books.google.com/books?id=wGUECAAAQBAJ|year=2012|publisher=Springer Science & Business Media|isbn=978-1-4613-8190-7|page=61}}</ref><ref>{{cite book|author1=Donald L. Snyder|author2=Michael I. Miller|title=Random Point Processes in Time and Space|url=https://books.google.com/books?id=c_3UBwAAQBAJ|year=2012|publisher=Springer Science & Business Media|isbn=978-1-4612-3166-0|page=27}}</ref><ref name="Bremaud2013page253">{{cite book|author=Pierre Bremaud|title=Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues|url=https://books.google.com/books?id=jrPVBwAAQBAJ|year=2013|publisher=Springer Science & Business Media|isbn=978-1-4757-3124-8|page=253}}</ref>

===Martingale===
{{Main|Martingale (probability theory)}}
A martingale is a discrete-time or continuous-time stochastic process with the property that, at every instant, given the current value and all the past values of the process, the conditional expectation of every future value is equal to the current value. In discrete time, if this property holds for the next value, then it holds for all future values. The exact mathematical definition of a martingale requires two other conditions coupled with the mathematical concept of a filtration, which is related to the intuition of increasing available information as time passes. Martingales are usually defined to be real-valued,<ref name="Klebaner2005page65">{{cite book|author=Fima C. Klebaner|title=Introduction to Stochastic Calculus with Applications|url=https://books.google.com/books?id=JYzW0uqQxB0C|year=2005|publisher=Imperial College Press|isbn=978-1-86094-555-7|page=65}}</ref><ref name="KaratzasShreve2014page11">{{cite book|author1=Ioannis Karatzas|author2=Steven Shreve|title=Brownian Motion and Stochastic Calculus|url=https://books.google.com/books?id=w0SgBQAAQBAJ&pg=PT5|year=1991|publisher=Springer|isbn=978-1-4612-0949-2|page=11}}</ref><ref name="Williams1991page93">{{cite book|author=David Williams|title=Probability with Martingales|url=https://books.google.com/books?id=e9saZ0YSi-AC|year=1991|publisher=Cambridge University Press|isbn=978-0-521-40605-5|pages=93, 94}}</ref> but they can also be complex-valued<ref name="Doob1990page292">{{cite book|author=Joseph L. Doob|title=Stochastic processes|url=https://books.google.com/books?id=NrsrAAAAYAAJ|year=1990|publisher=Wiley|pages=292, 293}}</ref> or even more general.<ref name="Pisier2016">{{cite book|author=Gilles Pisier|title=Martingales in Banach Spaces|url=https://books.google.com/books?id=n3JNDAAAQBAJ&pg=PR4|year=2016|publisher=Cambridge University Press|isbn=978-1-316-67946-3}}</ref>

A symmetric random walk and a Wiener process (with zero drift) are both examples of martingales, respectively, in discrete and continuous time.<ref name="Klebaner2005page65"/><ref name="KaratzasShreve2014page11"/> For a [[sequence]] of [[independent and identically distributed]] random variables <math>X_1, X_2, X_3, \dots</math> with zero mean, the stochastic process formed from the successive partial sums <math>X_1,X_1+ X_2, X_1+ X_2+X_3, \dots</math> is a discrete-time martingale.<ref name="Steele2012page12">{{cite book|author=J. Michael Steele|title=Stochastic Calculus and Financial Applications|url=https://books.google.com/books?id=fsgkBAAAQBAJ&pg=PR4|year=2012|publisher=Springer Science & Business Media|isbn=978-1-4684-9305-4|pages=12, 13}}</ref> In this aspect, discrete-time martingales generalize the idea of partial sums of independent random variables.<ref name="HallHeyde2014page2">{{cite book|author1=P. Hall|author2-link=Chris Heyde|author2=C. C. Heyde|title=Martingale Limit Theory and Its Application|url=https://books.google.com/books?id=gqriBQAAQBAJ&pg=PR10|year=2014|publisher=Elsevier Science|isbn=978-1-4832-6322-9|page=2}}</ref>

Martingales can also be created from stochastic processes by applying some suitable transformations, which is the case for the homogeneous Poisson process (on the real line) resulting in a martingale called the ''compensated Poisson process''.<ref name="KaratzasShreve2014page11"/> Martingales can also be built from other martingales.<ref name="Steele2012page12"/> For example, there are martingales based on the martingale the Wiener process, forming continuous-time martingales.<ref name="Klebaner2005page65"/><ref name="Steele2012page115">{{cite book|author=J. Michael Steele|title=Stochastic Calculus and Financial Applications|url=https://books.google.com/books?id=fsgkBAAAQBAJ&pg=PR4|year=2012|publisher=Springer Science & Business Media|isbn=978-1-4684-9305-4|page=115}}</ref>

Martingales mathematically formalize the idea of a 'fair game' where it is possible form reasonable expectations for payoffs,<ref name="Ross1996page295">{{cite book|author=Sheldon M. Ross|title=Stochastic processes|url=https://books.google.com/books?id=ImUPAQAAMAAJ|year=1996|publisher=Wiley|isbn=978-0-471-12062-9|page=295}}</ref> and they were originally developed to show that it is not possible to gain an 'unfair' advantage in such a game.<ref name="Steele2012page11"/> But now they are used in many areas of probability, which is one of the main reasons for studying them.<ref name="Williams1991page93"/><ref name="Steele2012page11">{{cite book|author=J. Michael Steele|title=Stochastic Calculus and Financial Applications|url=https://books.google.com/books?id=fsgkBAAAQBAJ&pg=PR4|year=2012|publisher=Springer Science & Business Media|isbn=978-1-4684-9305-4|page=11}}</ref><ref name="Kallenberg2002page96">{{cite book|author=Olav Kallenberg|title=Foundations of Modern Probability|url=https://books.google.com/books?id=L6fhXh13OyMC|year=2002|publisher=Springer Science & Business Media|isbn=978-0-387-95313-7|pages=96}}</ref> Many problems in probability have been solved by finding a martingale in the problem and studying it.<ref name="Steele2012page371">{{cite book|author=J. Michael Steele|title=Stochastic Calculus and Financial Applications|url=https://books.google.com/books?id=fsgkBAAAQBAJ&pg=PR4|year=2012|publisher=Springer Science & Business Media|isbn=978-1-4684-9305-4|page=371}}</ref> Martingales will converge, given some conditions on their moments, so they are often used to derive convergence results, due largely to [[martingale convergence theorem]]s.<ref name="HallHeyde2014page2"/><ref name="Steele2012page22">{{cite book|author=J. Michael Steele|title=Stochastic Calculus and Financial Applications|url=https://books.google.com/books?id=fsgkBAAAQBAJ&pg=PR4|year=2012|publisher=Springer Science & Business Media|isbn=978-1-4684-9305-4|page=22}}</ref><ref name="GrimmettStirzaker2001page336">{{cite book|author1=Geoffrey Grimmett|author2=David Stirzaker|title=Probability and Random Processes|url=https://books.google.com/books?id=G3ig-0M4wSIC|year=2001|publisher=OUP Oxford|isbn=978-0-19-857222-0|page=336}}</ref>

Martingales have many applications in statistics, but it has been remarked that its use and application are not as widespread as it could be in the field of statistics, particularly statistical inference.<ref name="GlassermanKou2006">{{cite journal|last1=Glasserman|first1=Paul|last2=Kou|first2=Steven|title=A Conversation with Chris Heyde|journal=Statistical Science|volume=21|issue=2|year=2006|pages=292, 293|issn=0883-4237|doi=10.1214/088342306000000088|arxiv=math/0609294|bibcode=2006math......9294G|s2cid=62552177}}</ref> They have found applications in areas in probability theory such as queueing theory and Palm calculus<ref name="BaccelliBremaud2013">{{cite book|author1=Francois Baccelli|author2=Pierre Bremaud|title=Elements of Queueing Theory: Palm Martingale Calculus and Stochastic Recurrences|url=https://books.google.com/books?id=DH3pCAAAQBAJ&pg=PR2|year=2013|publisher=Springer Science & Business Media|isbn=978-3-662-11657-9}}</ref> and other fields such as economics<ref name="HallHeyde2014pageX">{{cite book|author1=P. Hall|author2=C. C. Heyde|title=Martingale Limit Theory and Its Application|url=https://books.google.com/books?id=gqriBQAAQBAJ&pg=PR10|year= 2014|publisher=Elsevier Science|isbn=978-1-4832-6322-9|page=x}}</ref> and finance.<ref name="MusielaRutkowski2006"/>

===Lévy process===
{{Main|Lévy process}}
Lévy processes are types of stochastic processes that can be considered as generalizations of random walks in continuous time.<ref name="Applebaum2004page1337"/><ref name="Bertoin1998pageVIII">{{cite book|author=Jean Bertoin|title=Lévy Processes|url=https://books.google.com/books?id=ftcsQgMp5cUC&pg=PR8|year=1998|publisher=Cambridge University Press|isbn=978-0-521-64632-1|page=viii}}</ref> These processes have many applications in fields such as finance, fluid mechanics, physics and biology.<ref name="Applebaum2004page1336">{{cite journal|last1=Applebaum|first1=David|title=Lévy processes: From probability to finance and quantum groups|journal=Notices of the AMS|volume=51|issue=11|year=2004|pages=1336}}</ref><ref name="ApplebaumBook2004page69">{{cite book|author=David Applebaum|title=Lévy Processes and Stochastic Calculus|url=https://books.google.com/books?id=q7eDUjdJxIkC|year=2004|publisher=Cambridge University Press|isbn=978-0-521-83263-2|page=69}}</ref> The main defining characteristics of these processes are their stationarity and independence properties, so they were known as ''processes with stationary and independent increments''. In other words, a stochastic process <math>X</math> is a Lévy process if for <math>n</math> non-negatives numbers, <math>0\leq t_1\leq \dots \leq t_n</math>, the corresponding <math>n-1</math> increments
<div class="center"><math>
X_{t_2}-X_{t_1}, \dots ,  X_{t_n}-X_{t_{n-1}},
</math></div>
are all independent of each other, and the distribution of each increment only depends on the difference in time.<ref name="Applebaum2004page1337"/>

A Lévy process can be defined such that its state space is some abstract mathematical space, such as a [[Banach space]], but the processes are often defined so that they take values in Euclidean space. The index set is the non-negative numbers, so <math> I= [0,\infty) </math>, which gives the interpretation of time. Important stochastic processes such as the Wiener process, the homogeneous Poisson process (in one dimension), and [[subordinator (mathematics)|subordinators]] are all Lévy processes.<ref name="Applebaum2004page1337"/><ref name="Bertoin1998pageVIII"/>

===Random field===
{{Main|Random field}}
A random field is a collection of random variables indexed by a <math>n</math>-dimensional Euclidean space or some manifold. In general, a random field can be considered an example of a stochastic or random process, where the index set is not necessarily a subset of the real line.<ref name="AdlerTaylor2009page7"/> But there is a convention that an indexed collection of random variables is called a random field when the index has two or more dimensions.<ref name="GikhmanSkorokhod1969page1"/><ref name="Lamperti1977page1"/><ref name="KoralovSinai2007page171">{{cite book|author1=Leonid Koralov|author2=Yakov G. Sinai|title=Theory of Probability and Random Processes|url=https://books.google.com/books?id=tlWOphOFRgwC|year=2007|publisher=Springer Science & Business Media|isbn=978-3-540-68829-7|page=171}}</ref> If the specific definition of a stochastic process requires the index set to be a subset of the real line, then the random field can be considered as a generalization of stochastic process.<ref name="ApplebaumBook2004page19">{{cite book|author=David Applebaum|title=Lévy Processes and Stochastic Calculus|url=https://books.google.com/books?id=q7eDUjdJxIkC|year=2004|publisher=Cambridge University Press|isbn=978-0-521-83263-2|page=19}}</ref>

===Point process===
{{Main|Point process}}
A point process is a collection of points randomly located on some mathematical space such as the real line, <math>n</math>-dimensional Euclidean space, or more abstract spaces. Sometimes the term ''point process'' is not preferred, as historically the word ''process'' denoted an evolution of some system in time, so a point process is also called a '''random point field'''.<ref name="ChiuStoyan2013page109">{{cite book|author1=Sung Nok Chiu|author2=Dietrich Stoyan|author3=Wilfrid S. Kendall|author4=Joseph Mecke|title=Stochastic Geometry and Its Applications|url=https://books.google.com/books?id=825NfM6Nc-EC|year=2013|publisher=John Wiley & Sons|isbn=978-1-118-65825-3|page=109}}</ref> There are different interpretations of a point process, such a random counting measure or a random set.<ref name="ChiuStoyan2013page108">{{cite book|author1=Sung Nok Chiu|author2=Dietrich Stoyan|author3=Wilfrid S. Kendall|author4=Joseph Mecke|title=Stochastic Geometry and Its Applications|url=https://books.google.com/books?id=825NfM6Nc-EC|year=2013|publisher=John Wiley & Sons|isbn=978-1-118-65825-3|page=108}}</ref><ref name="Haenggi2013page10">{{cite book|author=Martin Haenggi|title=Stochastic Geometry for Wireless Networks|url=https://books.google.com/books?id=CLtDhblwWEgC|year=2013|publisher=Cambridge University Press|isbn=978-1-107-01469-5|page=10}}</ref> Some authors regard a point process and stochastic process as two different objects such that a point process is a random object that arises from or is associated with a stochastic process,<ref name="DaleyVere-Jones2006page194">{{cite book|author1=D.J. Daley|author2=D. Vere-Jones|title=An Introduction to the Theory of Point Processes: Volume I: Elementary Theory and Methods|url=https://books.google.com/books?id=6Sv4BwAAQBAJ|year=2006|publisher=Springer Science & Business Media|isbn=978-0-387-21564-8|page=194}}</ref><ref name="CoxIsham1980page3">{{cite book|first1=D. R.|last1=Cox|author1-link=David Cox (statistician)|first2=Valerie|last2=Isham|author2-link=Valerie Isham|title=Point Processes|at=[https://books.google.com/books?id=KWF2xY6s3PoC&pg=PA3 p. 3]|year=1980|publisher=CRC Press|isbn=978-0-412-21910-8|title-link= Point Processes}}</ref> though it has been remarked that the difference between point processes and stochastic processes is not clear.<ref name="CoxIsham1980page3"/>

Other authors consider a point process as a stochastic process, where the process is indexed by sets of the underlying space{{efn|In the context of point processes, the term "state space" can mean the space on which the point process is defined such as the real line,<ref name="Kingman1992page8">{{cite book|author=J. F. C. Kingman|title=Poisson Processes|url=https://books.google.com/books?id=VEiM-OtwDHkC|year=1992|publisher=Clarendon Press|isbn=978-0-19-159124-2|page=8}}</ref><ref name="MollerWaagepetersen2003page7">{{cite book|author1=Jesper Moller|author2=Rasmus Plenge Waagepetersen|title=Statistical Inference and Simulation for Spatial Point Processes|url=https://books.google.com/books?id=dBNOHvElXZ4C|year=2003|publisher=CRC Press|isbn=978-0-203-49693-0|page=7}}</ref> which corresponds to the index set in stochastic process terminology.}} on which it is defined, such as the real line or <math>n</math>-dimensional Euclidean space.<ref name="KarlinTaylor2012page31">{{cite book|author1=Samuel Karlin|author2=Howard E. Taylor|title=A First Course in Stochastic Processes|url=https://books.google.com/books?id=dSDxjX9nmmMC|year=2012|publisher=Academic Press|isbn=978-0-08-057041-9|page=31}}</ref><ref name="Schmidt2014page99">{{cite book|author=Volker Schmidt|title=Stochastic Geometry, Spatial Statistics and Random Fields: Models and Algorithms|url=https://books.google.com/books?id=brsUBQAAQBAJ&pg=PR5|date= 2014|publisher=Springer|isbn=978-3-319-10064-7|page=99}}</ref> Other stochastic processes such as renewal and counting processes are studied in the theory of point processes.<ref name="DaleyVere-Jones200">{{cite book|author1=D.J. Daley|author2=D. Vere-Jones|title=An Introduction to the Theory of Point Processes: Volume I: Elementary Theory and Methods|url=https://books.google.com/books?id=6Sv4BwAAQBAJ|year=2006|publisher=Springer Science & Business Media|isbn=978-0-387-21564-8}}</ref><ref name="CoxIsham1980page3" />