Editing Markov chain Monte Carlo (section)

===Reducing correlation===
While MCMC methods were created to address multi-dimensional problems better than generic Monte Carlo algorithms, when the number of dimensions rises they too tend to suffer the [[curse of dimensionality]]: regions of higher probability tend to stretch and get lost in an increasing volume of space that contributes little to the integral. One way to address this problem could be shortening the steps of the walker, so that it does not continuously try to exit the highest probability region, though this way the process would be highly autocorrelated and expensive (i.e. many steps would be required for an accurate result). More sophisticated methods such as [[Hamiltonian Monte Carlo]] and the [[Wang and Landau algorithm]] use various ways of reducing this autocorrelation, while managing to keep the process in the regions that give a higher contribution to the integral. These algorithms usually rely on a more complicated theory and are harder to implement, but they usually converge faster.

We outline several general strategies such as reparameterization, adaptive proposal tuning, parameter blocking, and overrelaxation that help reduce correlation and improve sampling efficiency within the standard MCMC framework.

====Reparameterization====
One way to reduce autocorrelation is to reformulate or reparameterize the statistical model so that the posterior geometry leads to more efficient sampling. By changing the coordinate system or using alternative variable definitions, one can often lessen correlations. For example, in [[Bayesian hierarchical modeling]], a non-centered parameterization can be used in place of the standard (centered) formulation to avoid extreme posterior correlations between latent and higher-level parameters. This involves expressing [[latent variables]] in terms of independent auxiliary variables, dramatically improving mixing. Such reparameterization strategies are commonly employed in both [[Gibbs sampling]] and [[Metropolis–Hastings algorithm]] to enhance convergence and reduce autocorrelation.<ref>{{cite journal
 | last1 = Papaspiliopoulos
 | first1 = Omiros
 | last2 = Roberts
 | first2 = Gareth O.
 | last3 = Sköld
 | first3 = Martin
 | title = A general framework for the parametrization of hierarchical models
 | journal = Statistical Science
 | volume = 22
 | issue = 1
 | pages = 59–73
 | year = 2007
 | publisher = Institute of Mathematical Statistics
 | doi = 10.1214/088342307000000014
}}</ref>

====Proposal tuning and adaptation====
Another approach to reducing correlation is to improve the MCMC proposal mechanism. In [[Metropolis–Hastings algorithm]], step size tuning is critical: if the proposed steps are too small, the sampler moves slowly and produces highly correlated samples; if the steps are too large, many proposals are rejected, resulting in repeated values. Adjusting the proposal step size during an initial testing phase helps find a balance where the sampler explores the space efficiently without too many rejections. 

Adaptive MCMC methods modify proposal distributions based on the chain's past samples. For instance, adaptive metropolis algorithm updates the Gaussian proposal distribution using the full information accumulated from the chain so far, allowing the proposal to adapt over time.<ref>{{cite journal
 | last1  = Haario | first1 = Heikki
 | last2  = Saksman | first2 = Eero
 | last3  = Tamminen | first3 = Johanna
 | title  = An adaptive Metropolis algorithm
 | journal= Bernoulli
 | volume = 7 | issue = 2 | pages = 223–242
 | year   = 2001
 | doi    = 10.2307/3318737
 | jstor = 3318737
 | url    = https://www.researchgate.net/publication/38322292
}}</ref>

====Parameter blocking====
Parameter blocking is a technique that reduces autocorrelation in MCMC by updating parameters jointly rather than one at a time. When parameters exhibit strong posterior correlations, one-by-one updates can lead to poor mixing and slow exploration of the target distribution. By identifying and sampling blocks of correlated parameters together, the sampler can more effectively traverse high-density regions of the posterior.

Parameter blocking is commonly used in both Gibbs sampling and Metropolis–Hastings algorithms. In blocked Gibbs sampling, entire groups of variables are updated conditionally at each step.<ref>Óli Páll Geirsson, Birgir Hrafnkelsson, and Helgi Sigurðarson (2015). "A Block Gibbs Sampling Scheme for Latent Gaussian Models." arXiv preprint [arXiv:1506.06285](https://arxiv.org/abs/1506.06285).</ref> In Metropolis–Hastings, multivariate proposals enable joint updates (i.e., updates of multiple parameters at once using a vector-valued proposal distribution, typically a multivariate Gaussian), though they often require careful tuning of the proposal covariance matrix.<ref>Siddhartha Chib and Srikanth Ramamurthy (2009). "Tailored Randomized Block MCMC Methods with Application to DSGE Models." *Journal of Econometrics*, 155(1), 19–38. [https://doi.org/10.1016/j.jeconom.2009.08.003 doi:10.1016/j.jeconom.2009.08.003]</ref>

====Overrelaxation====
Overrelaxation is a technique to reduce autocorrelation between successive samples by proposing new samples that are negatively correlated with the current state. This helps the chain explore the posterior more efficiently, especially in high-dimensional Gaussian models or when using Gibbs sampling. The basic idea is to reflect the current sample across the conditional mean, producing proposals that retain the correct stationary distribution but with reduced serial dependence. Overrelaxation is particularly effective when combined with Gaussian conditional distributions, where exact reflection or partial overrelaxation can be analytically implemented.<ref>Piero Barone, Giovanni Sebastiani, and Jonathan Stander (2002). "Over-relaxation methods and coupled Markov chains for Monte Carlo simulation." ''Statistics and Computing'', 12(1), 17–26. [https://doi.org/10.1023/A:1013112103963 doi:10.1023/A:1013112103963]</ref>