Editing Markov chain Monte Carlo (section)

=== Irreducibility and Aperiodicity ===
Recall that in the discrete setting, a [[Markov chain]] is said to be ''irreducible'' if it is possible to reach any state from any other state in a finite number of steps with positive probability. However, in the continuous setting, point-to-point transitions have zero probability. In this case, '''&phi;-irreducibility''' generalizes [[irreducibility]] by using a reference measure &phi; on the measurable space <math>(\mathcal{X},\mathcal{B}(\mathcal{X}))</math>.

;Definition (φ-irreducibility)
Given a measure <math>\varphi</math> defined on <math>(\mathcal{X},\mathcal{B}(\mathcal{X}))</math>, the Markov chain <math>(X_n)</math> with transition kernel <math>K(x, y)</math> is '''φ-irreducible''' if, for every <math>A \in \mathcal{B}(\mathcal{X})</math> with <math>\varphi(A) > 0</math>, there exists <math>n</math> such that <math>K^n(x, A) > 0</math> for all <math>x \in \mathcal{X}</math> (Equivalently, <math>P_x(\tau_A < \infty) > 0</math>, here <math>\tau_A = \inf\{n \geq 1 ; X_n \in A\}</math> is the first <math>n</math> for which the chain enters the set <math>A</math>). 

This is a more general definition for [[irreducibility]] of a [[Markov chain]] in non-discrete state space. In the discrete case, an irreducible Markov chain is said to be ''aperiodic'' if it has period 1. Formally, the period of a state <math>\omega \in \mathcal{X}</math> is defined as:

:<math>
d(\omega) := \mathrm{gcd}\{m \geq 1 \,;\, K^m(\omega, \omega) > 0\}
</math>

For the general (non-discrete) case, we define aperiodicity in terms of small sets:

;Definition (Cycle length and small sets)
A '''&phi;-irreducible''' Markov chain <math>(X_n)</math> has a ''cycle of length d'' if there exists a small set <math>C</math>, an associated integer <math>M</math>, and a probability distribution <math>\nu_M</math> such that ''d'' is the [[greatest common divisor]] of:

:<math>
\{ m \geq 1 \,;\, \exists\, \delta_m > 0 \text{ such that } C \text{ is small for } \nu_m \geq \delta_m \nu_M \}.
</math>

A set <math>C</math> is called '''small''' if there exists <math>m \in \mathbb{N}^*</math> and a nonzero measure <math>\nu_m</math> such that:

:<math>
K^m(x, A) \geq \nu_m(A), \quad \forall x \in C,\, \forall A \in \mathcal{B}(\mathcal{X}).
</math>