Editing Gibbs sampling (section)

== Implementation ==
Gibbs sampling, in its basic incarnation, is a special case of the [[Metropolis–Hastings algorithm]].  The point of Gibbs sampling is that given a [[multivariate distribution]] it is simpler to sample from a conditional distribution than to [[marginal distribution|marginalize]] by integrating over a [[joint distribution]].  Suppose we want to obtain <math>k</math> samples of a <math>n</math>-dimensional random vector <math>\mathbf{X} = (X_1, \dots, X_n)</math>. We proceed iteratively:

* Begin with some initial value <math>\mathbf{X}^{(0)}</math>.
* Given a sample <math>\mathbf{X}^{(i)} = \left(x_1^{(i)}, \dots, x_n^{(i)}\right)</math>, to obtain the next sample <math>\mathbf{X}^{(i+1)} = \left(x_1^{(i+1)}, x_2^{(i+1)}, \dots, x_n^{(i+1)}\right)</math>, we can sample each component <math>x_j^{(i+1)}</math> from the distribution of <math>X_j</math>, conditioned on all the components sampled so far: we condition on <math>X_{\ell}^{(i+1)}</math> for all <math>\ell\leq j-1</math>, and  on <math>X_{\ell}^{(i)}</math> for <math>j+1\leq\ell\leq n</math>. In other words, we sample <math>x_j^{(i+1)}</math> according to the distribution <math>P\left(X_j=\cdot|X_1=x_1^{(i+1)}, \dots, X_{j-1}=x_{j-1}^{(i+1)}, X_{j+1}=x_{j+1}^{(i)}, \dots, X_n=x_n^{(i)}\right)</math>.