Editing Markov property

{{Short description|Memoryless property of a stochastic process}}
{{about|the property of a stochastic process|the class of properties of a finitely presented group|Adian–Rabin theorem}}
[[File:Wiener process 3d.png|thumb|A single realisation of three-dimensional [[Brownian motion]] for times 0 ≤ t ≤ 2. Brownian motion has the Markov property, as the displacement of the particle does not depend on its past displacements.]]

In [[probability theory]] and [[statistics]], the term '''Markov property''' refers to the [[memoryless]] property of a [[stochastic process]], which means that its future evolution is independent of its history. It is named after the [[Russia]]n [[mathematician]]  [[Andrey Markov]]. The term '''strong Markov property''' is similar to the Markov property, except that the meaning of "present" is defined in terms of a random variable known as a [[stopping time]].

The term '''Markov assumption''' is used to describe a model where the Markov property is assumed to hold, such as a [[hidden Markov model]].

A [[Markov random field]] extends this property to two or more dimensions or to random variables defined for an interconnected network of items.<ref>[[Yadolah Dodge|Dodge, Yadolah]]. (2006) ''The Oxford Dictionary of Statistical Terms'', [[Oxford University Press]]. {{isbn|0-19-850994-4}}</ref> An example of a model for such a field is the [[Ising model]].

A discrete-time stochastic process satisfying the Markov property is known as a [[Markov chain]].

==Introduction==

A stochastic process has the Markov property if the [[conditional probability distribution]] of future states of the process (conditional on both past and present values) depends only upon the present state; that is, given the present, the future does not depend on the past. A process with this property is said to be '''Markov''' or '''Markovian''' and known as a '''[[Markov process]]'''. Two famous classes of Markov process are the [[Markov chain]] and [[Brownian motion]].

Note that there is a subtle, often overlooked and very important point that is often missed in the plain English statement of the definition: the statespace of the process is constant through time. The conditional description involves a fixed "bandwidth". For example, without this restriction we could augment any process to one which includes the complete history from a given initial condition and it would be made to be Markovian. But the state space would be of increasing dimensionality over time and does not meet the definition.

==History==

{{Main|Markov chain#History}}

==Definition==

Let <math>(\Omega,\mathcal{F},P)</math> be a [[probability space]] with a [[Filtration (probability theory)|filtration]] <math>(\mathcal{F}_s,\ s \in I)</math>, for some ([[totally ordered]]) index set <math>I</math>; and let <math>(S,\mathcal{S})</math> be a [[measurable space]]. A <math>(S,\mathcal{S})</math>-valued stochastic process <math>X=\{X_t:\Omega \to S\}_{t\in I}</math> [[Adapted process|adapted to the filtration]] is said to possess the ''Markov property'' if, for each <math>A \in \mathcal{S}</math>  and each <math>s,t\in I</math> with <math>s<t</math>,

:<math>P(X_t \in A \mid \mathcal{F}_s) = P(X_t \in A\mid X_s).</math><ref>[[Rick Durrett|Durrett, Rick]]. ''Probability: Theory and Examples''. Fourth Edition. [[Cambridge University Press]], 2010.</ref>

In the case where <math>S</math> is a discrete set with the [[Sigma-algebra#Simple set-based examples|discrete sigma algebra]] and <math>I = \mathbb{N}</math>, this can be reformulated as follows:

:<math>P(X_{n+1}=x_{n+1}\mid X_n=x_n, \dots, X_1=x_1)=P(X_{n+1}=x_{n+1}\mid X_n=x_n) \text{ for all } n \in \mathbb{N}.</math>

==Alternative formulations==

Alternatively, the Markov property can be formulated as follows.

:<math>\operatorname{E}[f(X_t)\mid\mathcal{F}_s]=\operatorname{E}[f(X_t)\mid\sigma(X_s)]</math>

for all <math>t\geq s\geq 0</math> and <math>f:S\rightarrow \mathbb{R}</math> bounded and measurable.<ref>{{cite book | author=Øksendal, Bernt K. | author-link= Bernt Øksendal | title=Stochastic Differential Equations: An Introduction with Applications | publisher=Springer, Berlin | year=2003 | isbn=3-540-04758-1}}</ref>

==Strong Markov property==
Suppose that <math>X=(X_t:t\geq 0)</math> is a [[stochastic process]] on a [[probability space]] <math>(\Omega,\mathcal{F},P)</math> with [[natural filtration]] <math>\{\mathcal{F}_t\}_{t\geq 0}</math>. Then for any [[stopping time]] <math> \tau </math> on <math> \Omega </math>, we can define

:<math>\mathcal{F}_{\tau}=\{A \in \mathcal{F}:\forall t \geq 0, \{\tau\geq t\} \cap A \in \mathcal{F}_{t}\}</math>.

Then <math>X</math> is said to have the strong Markov property if, for each [[stopping time]] <math>\tau</math>, conditional on the event <math>\{\tau < \infty\}</math>, we have that for each <math>t\ge 0</math>, <math>X_{\tau + t}</math> is independent of <math>\mathcal{F}_{\tau}</math> given <math>X_\tau</math>.

The strong Markov property implies the ordinary Markov property since by taking the stopping time <math>\tau=t</math>, the ordinary Markov property can be deduced.<ref>Ethier, Stewart N. and [[Thomas G. Kurtz|Kurtz, Thomas G.]] ''Markov Processes: Characterization and Convergence''. Wiley Series in Probability and Mathematical Statistics, 1986, p. 158.</ref>
<!-- Hide for now ... name '''Markov-type property''' seems doubtful
==Markov Type Property==
A stochastic process has a '''Markov-type property''' if the process's [[random variable]]s determine a set of probabilities can be factored in a way that yields the '''Markov property'''. Useful in applied research, members of such classes{{Clarify|October 2009|date=October 2009}} defined by their mathematics or area of application{{Clarify|October 2009|date=October 2009}} are referred to as '''[[Markov random field]]s'''., and occur in many situations. The [[Ising model]] is a prototypical example. -->

==In forecasting==
In the fields of [[predictive modelling]] and [[probabilistic forecasting]], the Markov property is considered desirable since it may enable the  reasoning and resolution of the problem that otherwise would not be possible to be resolved because of its [[Intractability (complexity)|intractability]]. Such a model is known as a [[Markov model]].

==Examples==
Assume that an urn contains two red balls and one green ball. One ball was drawn yesterday, one ball was drawn today, and the final ball will be drawn tomorrow. All of the draws are "without replacement".

Suppose you know that today's ball was red, but you have no information about yesterday's ball. The chance that tomorrow's ball will be red is 1/2. That's because the only two remaining outcomes for this random experiment are:

{| class="wikitable"
|-
! align="center" | Day !! align="center" | Outcome 1 !! align="center" | Outcome 2
|-
| align="center" | Yesterday || align="center" | Red || align="center" | Green
|-
| align="center" | Today || align="center" | Red || align="center" | Red
|-
| align="center"| Tomorrow || align="center" | Green || align="center" | Red
|}

On the other hand, if you know that both today and yesterday's balls were red, then you are guaranteed to get a green ball tomorrow.

This discrepancy shows that the probability distribution for tomorrow's color depends not only on the present value, but is also affected by information about the past. This stochastic process of observed colors doesn't have the Markov property. Using the same experiment above, if sampling "without replacement" is changed to sampling "with replacement," the process of observed colors will have the Markov property.<ref>{{cite web|url=https://math.stackexchange.com/q/89394 |title=Example of a stochastic process which does not have the Markov property |publisher=[[Stack Exchange]] | access-date= 2020-07-07}}</ref>

An application of the Markov property in a generalized form is in [[Markov chain Monte Carlo]] computations in the context of [[Bayesian statistics]].

==See also==
*[[Causal Markov condition]]
*[[Chapman–Kolmogorov equation]]
*[[Hysteresis]]
*[[Markov blanket]]
*[[Markov chain]]
*[[Markov decision process]]
*[[Markov model]]

== References ==

{{Reflist}}

[[Category:Markov models]]
[[Category:Markov processes]]