Editing Markov chain (section)

== Special types of Markov chains ==

=== Markov model ===
{{Main|Markov model}}
Markov models are used to model changing systems. There are 4 main types of models, that generalize Markov chains depending on whether every sequential state is observable or not, and whether the system is to be adjusted on the basis of observations made:
{| class="wikitable" style="border-spacing: 2px; border: 1px solid darkgray;"
!
!System state is fully observable
!System state is partially observable
|-
!System is autonomous
|Markov chain
|[[Hidden Markov model]]
|-
!System is controlled
|[[Markov decision process]]
|[[Partially observable Markov decision process]]
|}

===Bernoulli scheme===
{{Main|Bernoulli scheme}}
A [[Bernoulli scheme]] is a special case of a Markov chain where the transition probability matrix has identical rows, which means that the next state is independent of even the current state (in addition to being independent of the past states). A Bernoulli scheme with only two possible states is known as a [[Bernoulli process]].

Note, however, by the [[Ornstein isomorphism theorem]], that every aperiodic and irreducible Markov chain is isomorphic to a Bernoulli scheme;<ref name="nicol">
Matthew Nicol and Karl Petersen, (2009) "[https://www.math.uh.edu/~nicol/pdffiles/petersen.pdf Ergodic Theory: Basic Examples and Constructions]",
''Encyclopedia of Complexity and Systems Science'', Springer https://doi.org/10.1007/978-0-387-30440-3_177
</ref> thus, one might equally claim that Markov chains are a "special case" of Bernoulli schemes. The isomorphism generally requires a complicated recoding. The isomorphism theorem is even a bit stronger: it states that ''any'' [[stationary stochastic process]] is isomorphic to a Bernoulli scheme; the Markov chain is just one such example.

===Subshift of finite type===
{{Main|Subshift of finite type}}
When the Markov matrix is replaced by the [[adjacency matrix]] of a [[finite graph]], the resulting shift is termed a '''topological Markov chain''' or a '''subshift of finite type'''.<ref name="nicol"/> A Markov matrix that is compatible with the adjacency matrix can then provide a [[Measure (mathematics)|measure]] on the subshift. Many chaotic [[dynamical system]]s are isomorphic to topological Markov chains; examples include [[diffeomorphism]]s of [[closed manifold]]s, the [[Thue–Morse sequence|Prouhet–Thue–Morse system]], the [[Chacon system]], [[sofic system]]s, [[context-free system]]s and [[block-coding system]]s.<ref name="nicol"/>