Editing Bayes factor (section)

{{Short description|Statistical factor used to compare competing hypotheses}}

{{Bayesian statistics}}
The '''Bayes factor''' is a ratio of two competing [[statistical model]]s represented by their [[marginal likelihood|evidence]], and is used to quantify the support for one model over the other.<ref>{{cite journal |last1=Morey |first1=Richard D. |first2=Jan-Willem |last2=Romeijn |first3=Jeffrey N. |last3=Rouder |title=The philosophy of Bayes factors and the quantification of statistical evidence |journal=Journal of Mathematical Psychology |volume=72 |year=2016 |pages=6–18 |doi=10.1016/j.jmp.2015.11.001 |doi-access=free }}</ref> The models in question can have a common set of parameters, such as a [[null hypothesis]] and an alternative, but this is not necessary; for instance, it could also be a non-linear model compared to its [[linear approximation]]. The Bayes factor can be thought of as a Bayesian analog to the [[likelihood-ratio test]], although it uses the integrated (i.e., marginal) likelihood rather than the maximized likelihood. As such, both quantities only coincide under simple hypotheses (e.g., two specific parameter values).<ref>{{cite book |first1=Emmanuel |last1=Lesaffre |last2=Lawson |first2=Andrew B. |title=Bayesian Biostatistics |location=Somerset |publisher=John Wiley & Sons |year=2012 |isbn=978-0-470-01823-1 |chapter=Bayesian hypothesis testing |pages=72–78 |doi=10.1002/9781119942412.ch3 }}</ref> Also, in contrast with [[null hypothesis significance testing]], Bayes factors support evaluation of evidence ''in favor'' of a null hypothesis, rather than only allowing the null to be rejected or not rejected.<ref>{{cite journal |first1=Alexander |last1=Ly |first2=Angelika |last2=Stefan |first3=Johnny |last3=van Doorn |first4=Fabian |last4=Dablander |display-authors=1 |title=The Bayesian Methodology of Sir Harold Jeffreys as a Practical Alternative to the ''P'' Value Hypothesis Test |journal=Computational Brain & Behavior |volume=3 |issue= 2|pages=153–161 |year=2020 |doi=10.1007/s42113-019-00070-x  |doi-access=free |hdl=2066/226717 |hdl-access=free }}</ref>

Although conceptually simple, the computation of the Bayes factor can be challenging depending on the complexity of the model and the hypotheses.<ref>{{cite journal |first1=Fernando |last1=Llorente |first2=Luca |last2=Martino |first3=David |last3=Delgado |first4=Javier |display-authors=1 |last4=Lopez-Santiago |title=Marginal likelihood computation for model selection and hypothesis testing: an extensive review |journal=SIAM Review |volume=to appear |year=2023 |issue= |pages= 3–58|doi=10.1137/20M1310849 |arxiv=2005.08334 |s2cid=210156537 }}</ref> Since closed-form expressions of the marginal likelihood are generally not available, numerical approximations based on [[Markov chain Monte Carlo|MCMC samples]] have been suggested.<ref>{{cite book |first=Peter |last=Congdon |chapter=Estimating model probabilities or marginal likelihoods in practice |pages=38–40 |title=Applied Bayesian Modelling |publisher=Wiley |edition=2nd |year=2014 |isbn=978-1-119-95151-3 }}</ref> A widely used approach is the method proposed by Chib (1995).<ref>{{cite journal |last=Chib |first=Siddhartha |year=1995 |title=Marginal Likelihood from the Gibbs Output |journal=Journal of the American Statistical Association |volume=90 |issue=432 |pages=1313–1321 |doi=10.2307/2291521 }}</ref> Chib and Jeliazkov (2001) later extended this method to handle cases where Metropolis-Hastings samplers are used.<ref>{{cite journal |last1=Chib |first1=Siddhartha |last2=Jeliazkov |first2=Ivan |year=2001 |title=Marginal Likelihood from the Metropolis–Hastings Output |journal=Journal of the American Statistical Association |volume=96 |issue=453 |pages=270–281 |doi=10.1198/016214501750332848 }}</ref> For certain special cases, simplified algebraic expressions can be derived; for instance, the Savage–Dickey density ratio in the case of a precise (equality constrained) hypothesis against an unrestricted alternative.<ref>{{cite book |first=Gary |last=Koop |title=Bayesian Econometrics |location=Somerset |publisher=John Wiley & Sons |year=2003 |isbn=0-470-84567-8 |chapter=Model Comparison: The Savage–Dickey Density Ratio |pages=69–71 }}</ref><ref>{{cite journal |first1=Eric-Jan |last1=Wagenmakers |first2=Tom |last2=Lodewyckx |first3=Himanshu |last3=Kuriyal |first4=Raoul |last4=Grasman |title=Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method |journal=Cognitive Psychology |volume=60 |issue=3 |year=2010 |pages=158–189 |doi=10.1016/j.cogpsych.2009.12.001 |pmid=20064637 |s2cid=206867662 |url=http://www.ejwagenmakers.com/2010/WagenmakersEtAlCogPsy2010.pdf }}</ref> Another approximation, derived by applying [[Laplace's approximation]] to the integrated likelihoods, is known as the [[Bayesian information criterion]] (BIC);<ref>{{cite book |first1=Joseph G. |last1=Ibrahim |first2=Ming-Hui |last2=Chen |first3=Debajyoti |last3=Sinha |chapter=Model Comparison |series=Springer Series in Statistics |title=Bayesian Survival Analysis |location=New York |publisher=Springer |year=2001 |isbn=0-387-95277-2 |doi=10.1007/978-1-4757-3447-8_6 |pages=246–254 }}</ref> in large data sets the Bayes factor will approach the BIC as the influence of the priors wanes. In small data sets, priors generally matter and must not be [[improper prior|improper]] since the Bayes factor will be undefined if either of the two integrals in its ratio is not finite.