Editing Examples of Markov chains (section)

=== Stock market ===
[[File:Finance_Markov_chain_example_state_space.svg|right|thumb|400x400px|Using a directed graph, the probabilities of the possible states a hypothetical stock market can exhibit is represented. The matrix on the left shows how probabilities corresponding to different states can be arranged in matrix form.]]
A [[state diagram]] for a simple example is shown in the figure on the right, using a directed graph to picture the [[State transition|state transitions]]. The states represent whether a hypothetical stock market is exhibiting a [[bull market]], [[bear market]], or stagnant market trend during a given week. According to the figure, a bull week is followed by another bull week 90% of the time, a bear week 7.5% of the time, and a stagnant week the other 2.5% of the time. Labeling the state space {1&nbsp;=&nbsp;bull, 2&nbsp;=&nbsp;bear, 3&nbsp;=&nbsp;stagnant} the transition matrix for this example is

: <math>P = \begin{bmatrix}
0.9 & 0.075 & 0.025 \\
0.15 & 0.8 & 0.05 \\
0.25 & 0.25 & 0.5
\end{bmatrix}.</math>

The distribution over states can be written as a [[stochastic row vector]] {{mvar|x}} with the relation {{math|1=''x''<sup>(''n''&nbsp;+&nbsp;1)</sup>&nbsp;=&nbsp;''x''<sup>(''n'')</sup>''P''}}. So if at time {{mvar|n}} the system is in state {{math|''x''<sup>(''n'')</sup>}}, then three time periods later, at time {{math|''n''&nbsp;+&nbsp;3}} the distribution is

: <math>\begin{align}
x^{(n+3)} &= x^{(n+2)} P = \left( x^{(n+1)} P \right) P \\\\
&= x^{(n+1)} P^2= \left( x^{(n)} P \right) P^2\\
&= x^{(n)} P^3 \\
\end{align}</math>

In particular, if at time {{mvar|n}} the system is in state 2&nbsp;(bear), then at time {{math|''n''&nbsp;+&nbsp;3}} the distribution is

: <math>\begin{align}
x^{(n+3)} &= \begin{bmatrix} 0 & 1 & 0 \end{bmatrix}
\begin{bmatrix}
0.9 & 0.075 & 0.025 \\
0.15 & 0.8 & 0.05 \\
0.25 & 0.25 & 0.5
\end{bmatrix}^3 \\[5pt]
&= \begin{bmatrix} 0 & 1 & 0 \end{bmatrix} \begin{bmatrix}
0.7745 & 0.17875 & 0.04675 \\
0.3575 & 0.56825 & 0.07425 \\
0.4675 & 0.37125 & 0.16125 \\
\end{bmatrix} \\[5pt]
& = \begin{bmatrix} 0.3575 & 0.56825 & 0.07425 \end{bmatrix}.
\end{align}</math>

Using the transition matrix it is possible to calculate, for example, the long-term fraction of weeks during which the market is stagnant, or the average number of weeks it will take to go from a stagnant to a bull market. Using the transition probabilities, the steady-state probabilities indicate that 62.5% of weeks will be in a bull market, 31.25% of weeks will be in a bear market and 6.25% of weeks will be stagnant, since:

: <math>\lim_{N\to \infty } \, P^N=
\begin{bmatrix}
0.625 & 0.3125 & 0.0625 \\
0.625 & 0.3125 & 0.0625 \\
0.625 & 0.3125 & 0.0625
\end{bmatrix}</math>

A thorough development and many examples can be found in the on-line monograph Meyn & Tweedie 2005.<ref name="MCSS">S. P. Meyn and R.L. Tweedie, 2005. [http://probability.ca/MT/BOOK.pdf Markov Chains and Stochastic Stability] {{webarchive|url=https://web.archive.org/web/20130903184125/http://probability.ca/MT/BOOK.pdf|date=2013-09-03}}</ref>

A [[finite-state machine]] can be used as a representation of a Markov chain. Assuming a sequence of [[Independent and identically distributed random variables|independent and identically distributed]] input signals (for example, symbols from a binary alphabet chosen by coin tosses), if the machine is in state ''y'' at time ''n'', then the probability that it moves to state ''x'' at time ''n''&nbsp;+&nbsp;1 depends only on the current state.