Editing Bayesian statistics (section)

==Bayesian methods==

The general set of statistical techniques can be divided into a number of activities, many of which have special Bayesian versions.

=== Bayesian inference ===
{{Main|Bayesian inference}}
Bayesian inference refers to [[statistical inference]] where uncertainty in inferences is quantified using probability.<ref>{{Cite journal |last=Lee|first=Se Yoon|  title = Gibbs sampler and coordinate ascent variational inference: A set-theoretical review|journal=Communications in Statistics - Theory and Methods|year=2021|volume=51 |issue=6 |pages=1549–1568 |doi=10.1080/03610926.2021.1921214|arxiv=2008.01006|s2cid=220935477 }}</ref> In classical [[frequentist inference]], model [[parameter]]s and hypotheses are considered to be fixed. Probabilities are not assigned to parameters or hypotheses in frequentist inference. For example, it would not make sense in frequentist inference to directly assign a probability to an event that can only happen once, such as the result of the next flip of a fair coin. However, it would make sense to state that the proportion of heads [[Law of large numbers|approaches one-half]] as the number of coin flips increases.<ref name="wakefield2013">{{cite book |last1=Wakefield |first1=Jon |title=Bayesian and frequentist regression methods |date=2013 |publisher=Springer |location=New York, NY |isbn=978-1-4419-0924-4}}</ref>

[[Statistical models]] specify a set of statistical assumptions and processes that represent how the sample data are generated. Statistical models have a number of parameters that can be modified. For example, a coin can be represented as samples from a [[Bernoulli distribution]], which models two possible outcomes. The Bernoulli distribution has a single parameter equal to the probability of one outcome, which in most cases is the probability of landing on heads. Devising a good model for the data is central in Bayesian inference. In most cases, models only approximate the true process, and may not take into account certain factors influencing the data.<ref name="bda" /> In Bayesian inference, probabilities can be assigned to model parameters. Parameters can be represented as [[random variable]]s. Bayesian inference uses Bayes' theorem to update probabilities after more evidence is obtained or known.<ref name="bda" /><ref name="congdon2014">{{cite book |last1=Congdon |first1=Peter |title=Applied Bayesian modelling |date=2014 |publisher=Wiley |isbn=978-1119951513 |edition=2nd}}</ref> Furthermore, Bayesian methods allow for placing priors on entire models and calculating their posterior probabilities using Bayes' theorem. These posterior probabilities 
are proportional to the product of the prior and the marginal likelihood, 
where the marginal likelihood is the integral of the sampling density over the prior distribution 
of the parameters. In complex models, marginal likelihoods are 
generally computed numerically.<ref name="chib1995">{{cite journal 
|last=Chib 
|first=Siddhartha 
|title=Marginal Likelihood from the Gibbs Output 
|journal=Journal of the American Statistical Association 
|year=1995 
|volume=90 
|issue=432 
|pages=1313-1321 
|doi=10.1080/01621459.1995.10476635}}</ref>

===Statistical modeling===
The formulation of [[statistical model]]s using Bayesian statistics has the identifying feature of requiring the specification of [[prior distribution]]s for any unknown parameters. Indeed, parameters of prior distributions may themselves have prior distributions, leading to [[Bayesian hierarchical modeling]],<ref name="KruschkeVanpaemel2015">{{cite book |last1=Kruschke|first1=J K|author-link1=John K. Kruschke |last2=Vanpaemel |first2=W |chapter=Bayesian Estimation in Hierarchical Models |pages=279–299 |title=The Oxford Handbook of Computational and Mathematical Psychology |editor-last1=Busemeyer |editor-first1=J R |editor-last2=Wang |editor-first2=Z |editor-last3=Townsend |editor-first3=J T |editor-last4=Eidels |editor-first4=A |year=2015 |publisher=Oxford University Press |url=https://jkkweb.sitehost.iu.edu/articles/KruschkeVanpaemel2015.pdf}}</ref><ref name=":bmdl">Hajiramezanali, E. & Dadaneh, S. Z. & Karbalayghareh, A. & Zhou, Z. & Qian, X. Bayesian multi-domain learning for cancer subtype discovery from next-generation sequencing count data. 32nd Conference on Neural Information Processing Systems (NIPS 2018), Montréal, Canada. {{ArXiv|1810.09433}}</ref><ref>{{Cite journal |last1=Lee|first1=Se Yoon |first2=Bani|last2=Mallick|  title = Bayesian Hierarchical Modeling: Application Towards Production Results in the Eagle Ford Shale of South Texas|journal=Sankhya B|year=2021|volume=84 |pages=1–43 |doi=10.1007/s13571-020-00245-8|doi-access=}}</ref> also known as multi-level modeling. A special case is [[Bayesian networks]].

For conducting a Bayesian statistical analysis, best practices are discussed by van de Schoot et al.<ref name="vandeShootEtAl2021">{{cite journal|last1=van de Schoot |first1=Rens|last2=Depaoli |first2=Sarah |last3=King |first3=Ruth |last4=Kramer |first4=Bianca |last5=Märtens |first5=Kaspar |last6=Tadesse |first6=Mahlet G. |last7=Vannucci |first7=Marina |last8=Gelman |first8=Andrew |last9=Veen |first9=Duco |last10=Willemsen |first10=Joukje |last11=Yau |first11=Christopher |title=Bayesian statistics and modelling|journal=Nature Reviews Methods Primers|date=January 14, 2021|volume=1|number=1|pages=1–26|doi=10.1038/s43586-020-00001-2|hdl=1874/415909 |s2cid=234108684 |url=https://osf.io/wdtmc/|hdl-access=free }}</ref> 

For reporting the results of a Bayesian statistical analysis, Bayesian analysis reporting guidelines (BARG) are provided in an open-access article by [[John K. Kruschke]].<ref name="Kruschke2021BARG">{{cite journal|last=Kruschke|first=J K|author-link=John K. Kruschke|title=Bayesian Analysis Reporting Guidelines|journal=Nature Human Behaviour|date=Aug 16, 2021|volume=5|issue=10 |pages=1282–1291|doi=10.1038/s41562-021-01177-7|pmid=34400814 |pmc=8526359 }}</ref>

===Design of experiments===
The [[Bayesian design of experiments]] includes a concept called 'influence of prior beliefs'. This approach uses [[sequential analysis]] techniques to include the outcome of earlier experiments in the design of the next experiment. This is achieved by updating 'beliefs' through the use of prior and [[posterior distribution]]. This allows the design of experiments to make good use of resources of all types. An example of this is the [[multi-armed bandit problem]].

===Exploratory analysis of Bayesian models===

Exploratory analysis of Bayesian models is an adaptation or extension of the [[exploratory data analysis]] approach to the needs and peculiarities of Bayesian modeling. In the words of Persi Diaconis:<ref>Diaconis, Persi (2011) Theories of Data Analysis: From Magical Thinking Through Classical Statistics. John Wiley & Sons, Ltd 2:e55 {{doi|10.1002/9781118150702.ch1}}</ref> 
{{Quote|Exploratory data analysis seeks to reveal structure, or simple descriptions in data. We look at numbers or graphs and try to find patterns. We pursue leads suggested by background information, imagination, patterns perceived, and experience with other data analyses}}

The [[Bayesian inference|inference process]] generates a posterior distribution, which has a central role in Bayesian statistics, together with other distributions like the posterior predictive distribution and the prior predictive distribution. The correct visualization, analysis, and interpretation of these distributions is key to properly answer the questions that motivate the inference process.<ref>{{cite journal |doi=10.21105/joss.01143 |title=ArviZ a unified library for exploratory analysis of Bayesian models in Python |year=2019 |last1=Kumar |first1=Ravin |last2=Carroll |first2=Colin |last3=Hartikainen |first3=Ari |last4=Martin |first4=Osvaldo |journal=Journal of Open Source Software |volume=4 |issue=33 |page=1143 |bibcode=2019JOSS....4.1143K |doi-access=free |hdl=11336/114615 |hdl-access=free }}</ref>

When working with Bayesian models there are a series of related tasks that need to be addressed besides inference itself:

* Diagnoses of the quality of the inference, this is needed when using numerical methods such as [[Markov chain Monte Carlo]] techniques
* Model criticism, including evaluations of both model assumptions and model predictions
* Comparison of models, including model selection or model averaging
* Preparation of the results for a particular audience

All these tasks are part of the Exploratory analysis of Bayesian models approach and successfully performing them is central to the iterative and interactive modeling process. These tasks require both numerical and visual summaries.<ref>{{cite journal |arxiv=1709.01449 |doi=10.1111/rssa.12378 |title=Visualization in Bayesian workflow |year=2019 |last1=Gabry |first1=Jonah |last2=Simpson |first2=Daniel |last3=Vehtari |first3=Aki |last4=Betancourt |first4=Michael |last5=Gelman |first5=Andrew |s2cid=26590874 |journal=Journal of the Royal Statistical Society, Series A (Statistics in Society) |volume=182 |issue=2 |pages=389–402 }}</ref><ref>{{cite journal |arxiv=1903.08008 |last1=Vehtari |first1=Aki |last2=Gelman |first2=Andrew |last3=Simpson |first3=Daniel |last4=Carpenter |first4=Bob |last5=Bürkner |first5=Paul-Christian |title=Rank-Normalization, Folding, and Localization: An Improved Rˆ for Assessing Convergence of MCMC (With Discussion) |journal=Bayesian Analysis |year=2021 |volume=16 |issue=2 |page=667 |doi=10.1214/20-BA1221 |bibcode=2021BayAn..16..667V |s2cid=88522683 }}</ref><ref name="Martin2018">{{cite book|url=https://books.google.com/books?id=1Z2BDwAAQBAJ|title=Bayesian Analysis with Python: Introduction to statistical modeling and probabilistic programming using PyMC3 and ArviZ|last1=Martin|first1=Osvaldo|date=2018|publisher=Packt Publishing Ltd|isbn=9781789341652|language=en}}</ref>