Editing Bayesian inference (section)

==In frequentist statistics and decision theory==

A [[statistical decision theory|decision-theoretic]] justification of the use of Bayesian inference was given by [[Abraham Wald]], who proved that every unique Bayesian procedure is [[admissible decision rule|admissible]]. Conversely, every [[admissible decision rule|admissible]] statistical procedure is either a Bayesian procedure or a limit of Bayesian procedures.<ref name="Bickel & Doksum 2001, page 32">Bickel & Doksum (2001, p. 32)</ref>

Wald characterized admissible procedures as Bayesian procedures (and limits of Bayesian procedures), making the Bayesian formalism a central technique in such areas of [[frequentist inference]] as [[parameter estimation]], [[hypothesis testing]], and computing [[confidence intervals]].<ref>{{cite journal|doi=10.1214/aoms/1177700051|author=Kiefer, J. |author2=Schwartz R. |title=Admissible Bayes Character of T<sup>2</sup>-, R<sup>2</sup>-, and Other Fully Invariant Tests for Multivariate Normal Problems|journal=Annals of Mathematical Statistics| volume=36|issue=3 |year=1965|pages=747–770|author-link=Jack Kiefer (mathematician) |doi-access=free}}</ref><ref>{{cite journal |doi= 10.1214/aoms/1177697822| author=Schwartz, R.|title=Invariant Proper Bayes Tests for Exponential Families |journal=Annals of Mathematical Statistics| volume=40 |year=1969| pages=270–283|doi-access=free}}</ref><ref>{{cite journal|doi=10.1214/aos/1176345877|author1=Hwang, J. T.  |author2=Casella, George  |name-list-style=amp |title=Minimax Confidence Sets for the Mean of a Multivariate Normal Distribution|journal=Annals of Statistics| volume=10|issue=3 | pages=868–881|year=1982|url= http://ecommons.cornell.edu/bitstream/1813/32852/1/BU-750-M.pdf|doi-access=free}}</ref> For example:
* "Under some conditions, all admissible procedures are either Bayes procedures or limits of Bayes procedures (in various senses). These remarkable results, at least in their original form, are due essentially to Wald. They are useful because the property of being Bayes is easier to analyze than admissibility."<ref name="Bickel & Doksum 2001, page 32"/>
* "In decision theory, a quite general method for proving admissibility consists in exhibiting a procedure as a unique Bayes solution."<ref>{{cite book|author=Lehmann, Erich| title=Testing Statistical Hypotheses|edition=Second|year=1986| author-link=Erich Leo Lehmann}} (see p. 309 of Chapter 6.7 "Admissibility", and pp. 17–18 of Chapter 1.8 "Complete Classes"</ref>
*"In the first chapters of this work, prior distributions with finite support and the corresponding Bayes procedures were used to establish some of the main theorems relating to the comparison of experiments. Bayes procedures with respect to more general prior distributions have played a very important role in the development of statistics, including its asymptotic theory." "There are many problems where a glance at posterior distributions, for suitable priors, yields immediately interesting information. Also, this technique can hardly be avoided in sequential analysis."<ref>{{cite book|last=Le Cam|first= Lucien|title=Asymptotic Methods in Statistical Decision Theory|year=1986|publisher=Springer-Verlag | isbn=978-0-387-96307-5|author-link=Lucien Le Cam}} (From "Chapter 12 Posterior Distributions and Bayes Solutions", p. 324)</ref>
*"A useful fact is that any Bayes decision rule obtained by taking a proper prior over the whole parameter space must be admissible"<ref>{{cite book |last1=Cox | first1 = D. R. | last2=Hinkley | first2 = D.V. |title=Theoretical Statistics |year=1974 | publisher=Chapman and Hall |isbn=978-0-04-121537-3 |page = 432 |author-link=David R. Cox }}</ref>
*"An important area of investigation in the development of admissibility ideas has been that of conventional sampling-theory procedures, and many interesting results have been obtained."<ref>{{cite book|last1=Cox | first1 = D. R. | last2=Hinkley | first2 = D.V. | title=Theoretical Statistics|year=1974 |publisher=Chapman and Hall|isbn=978-0-04-121537-3|page = 433|author-link=David R. Cox }})</ref>

===Model selection===
{{main|Bayesian model selection}}
{{see also|Bayesian information criterion}}
Bayesian methodology also plays a role in [[model selection]] where the aim is to select one model from a set of competing models that represents most closely the underlying process that generated the observed data. In Bayesian model comparison, the model with the highest [[posterior probability]] given the data is selected. The posterior probability of a model depends on the evidence, or [[marginal likelihood]], which reflects the probability that the data is generated by the model, and on the [[Prior probability|prior belief]] of the model. When two competing models are a priori considered to be equiprobable, the ratio of their posterior probabilities corresponds to the [[Bayes factor]]. Since Bayesian model comparison is aimed on selecting the model with the highest posterior probability, this methodology is also referred to as the maximum a posteriori (MAP) selection rule <ref>{{cite journal|first1= P.|last1= Stoica |first2 = Y.|last2 =Selen|journal = IEEE Signal Processing Magazine |date = 2004| title = A review of information criterion rules|doi=10.1109/MSP.2004.1311138|volume=21|issue=4|pages=36–47|s2cid= 17338979 }}</ref> or the MAP probability rule.<ref>{{cite journal|first1= J.|last1= Fatermans |first2 = S.|last2 =Van Aert |first3=A.J. |last3=den Dekker|journal = Ultramicroscopy |date = 2019|title = The maximum a posteriori probability rule for atom column detection from HAADF STEM images|doi=10.1016/j.ultramic.2019.02.003|volume=201|pages=81–91|pmid= 30991277 |arxiv=1902.05809| s2cid= 104419861 }}</ref>