Editing Frequentist probability

{{Short description|Interpretation of probability}}
{{Redirect|Statistical probability| the TV series episode|Statistical Probabilities}}
{{Use dmy dates|date=March 2020}}
[[Image:John Venn.jpg|thumb|upright=0.9|[[John Venn]], who provided a thorough exposition of frequentist probability in his book, ''The Logic of Chance''<ref name=Venn-1888/>]]

'''Frequentist probability''' or '''frequentism''' is an [[interpretation of probability]]; it defines an event's [[probability]] (the ''long-run probability'') as the [[limit of a sequence|limit]] of its [[Empirical probability|relative frequency]] in infinitely many [[Experiment (probability theory)|trials]].<ref>
{{cite book
 | last=Kaplan | first=D.
 | year=2014
 | title=Bayesian Statistics for the Social Sciences
 | publisher=Guilford Publications
 | series=Methodology in the Social Sciences
 | isbn=978-1-4625-1667-4
 | url=https://books.google.com/books?id=JFwKBAAAQBAJ&pg=PA4
 | access-date=2022-04-23
 | page=4
}}
</ref>
Probabilities can be found (in principle) by a repeatable objective process, as in repeated [[sampling (statistics)|sampling]] from the same [[population (statistics)|population]], and are thus ideally devoid of subjectivity. The continued use of frequentist methods in scientific inference, however, has been called into question.<ref>
{{cite journal
 |last = Goodman |first = Steven N.
 |year=1999
 |title = Toward evidence-based medical statistics. 1:&nbsp;The {{nobr|{{mvar|p}} value}} fallacy
 |journal = [[Annals of Internal Medicine]]
 |volume = 130 |issue = 12 |pages = 995–1004
 |pmid = 10383371 |s2cid = 7534212
 |doi = 10.7326/0003-4819-130-12-199906150-00008
}}
</ref><ref>
{{cite journal
 |last1 = Morey    |first1 = Richard D.    |last2 = Hoekstra |first2 = Rink
 |last3 = Rouder   |first3 = Jeffrey N.    |last4 = Lee      |first4 = Michael D.
 |last5 = Wagenmakers |first5 = Eric-Jan
 |year = 2016
 |title = The fallacy of placing confidence in confidence intervals
 |journal = Psychonomic Bulletin & Review
 |volume=23 |issue=1 |pages=103–123
 |doi = 10.3758/s13423-015-0947-8
 |pmid=26450628 |pmc=4742505
}}
</ref><ref>
{{cite journal
 |last = Matthews  |first = Robert
 |year = 2021
 |title = The {{mvar|p}}-value statement, five years on
 |journal = Significance
 |volume = 18  |issue = 2  |pages = 16–19
 |s2cid = 233534109
 |doi = 10.1111/1740-9713.01505
}}
</ref>

The development of the frequentist account was motivated by the problems and paradoxes of the previously dominant viewpoint, the [[Classical definition of probability|classical interpretation]]. In the classical interpretation, probability was defined in terms of the [[principle of indifference]], based on the natural symmetry of a problem, so, for example, the probabilities of dice games arise from the natural symmetric 6-sidedness of the cube. This classical interpretation stumbled at any statistical problem that has no natural symmetry for reasoning.

==Definition==
In the frequentist interpretation, probabilities are discussed only when dealing with well-defined random experiments. The set of all possible outcomes of a random experiment is called the sample space of the experiment. An event is defined as a particular subset of the sample space to be considered. For any given event, only one of two possibilities may hold: It occurs or it does not. The relative frequency of occurrence of an event, observed in a number of repetitions of the experiment, is a measure of the probability of that event. This is the core conception of probability in the frequentist interpretation.

A claim of the frequentist approach is that, as the number of trials increases, the change in the relative frequency will diminish. Hence, one can view a probability as the ''limiting value'' of the corresponding relative frequencies.

== Scope ==
The frequentist interpretation is a philosophical approach to the definition and use of probabilities; it is one of several such approaches. It does not claim to capture all connotations of the concept 'probable' in colloquial speech of natural languages.

As an interpretation, it is not in conflict with the mathematical axiomatization of probability theory; rather, it provides guidance for how to apply mathematical probability theory to real-world situations. It offers distinct guidance in the construction and design of practical experiments, especially when contrasted with the [[Bayesian probability|Bayesian interpretation]]. As to whether this guidance is useful, or is apt to mis-interpretation, has been a source of controversy. Particularly when the frequency interpretation of probability is mistakenly assumed to be the only possible basis for [[frequentist inference]]. So, for example, a list of mis-interpretations of the meaning of [[p-values]] accompanies the article on {{mvar|p}}-values; controversies are detailed in the article on [[Statistical hypothesis testing#Controversy|statistical hypothesis testing]]. The [[Jeffreys–Lindley paradox]] shows how different interpretations, applied to the same data set, can lead to different conclusions about the 'statistical significance' of a result.{{citation needed|date=April 2012}}

As [[William Feller|Feller]] notes:{{efn|
Feller's comment is a criticism of [[Pierre-Simon Laplace]]'s solution to the "tomorrow's sunrise" problem that used an alternative probability interpretation.
:
Despite [[Pierre-Simon Laplace|Laplace]]'s explicit and immediate disclaimer ''in the source'', based on Laplace's personal expertise in both astronomy and probability, two centuries of nattering criticism have followed.
}}
{{Blockquote|
There is no place in our system for speculations concerning the probability that the [[sunrise problem|sun will rise tomorrow]]. Before speaking of it we should have to agree on an (idealized) model which would presumably run along the lines ''"out of infinitely many worlds one is selected at random&nbsp;..."'' Little imagination is required to construct such a model, but it appears both uninteresting and meaningless.<ref name=Feller-1957>
{{cite book
 |first = W. |last = Feller  |author-link = William Feller
 |year = 1957
 |title = An Introduction to Probability Theory and Its Applications
 |volume = 1  |page = 4
}}
</ref>
}}

== History ==
{{main|History of probability}}
The frequentist view may have been foreshadowed by [[Aristotle]], in ''[[Rhetoric (Aristotle)|Rhetoric]]'',<ref name=keynesVIII>
{{cite book
 |last = Keynes |first = J.M. |author-link = John Maynard Keynes
 |title = A Treatise on Probability
 |year = 1921
 |chapter = Chapter&nbsp;VIII – The frequency theory of probability
}}
</ref>
when he wrote:
{{Quote|
the probable is that which for the most part happens — [[Aristotle]] [[Rhetoric (Aristotle)|''Rhetoric'']]<ref name=aristorhetor>
{{cite book
 |author = [[Aristotle]]
 |title = Rhetoric  |title-link = Rhetoric (Aristotle) 
 |at=Bk&nbsp;1, Ch&nbsp;2
}}
: discussed in
{{cite book
 |first = J. |last = Franklin
 |title = The Science of Conjecture: Evidence and probability before Pascal
 |year = 2001
 |publisher = The Johns Hopkins University Press
 |place = Baltimore, MD
 |ISBN = 0801865697
 |page = 110
}}
</ref>
}}

[[Siméon Denis Poisson|Poisson]] (1837) clearly distinguished between objective and subjective probabilities.<ref name=Gig89>
{{cite book
 |last1 = Gigerenzer |first1 = Gerd             |last2 = Swijtink |first2 = Porter
 |last3 = Daston     |first3 = Beatty           |last4 = Daston   |first4 = Krüger
 |year = 1989
 |title = The Empire of Chance : How probability changed science and everyday life
 |publisher = Cambridge University Press
 |isbn = 978-0-521-39838-1
 |location = Cambridge, UK / New York, NY
 |pages = 35–36, 45
}}
</ref>
Soon thereafter a flurry of nearly simultaneous publications by [[John Stuart Mill|Mill]], [[Robert Leslie Ellis|Ellis]] (1843)<ref name=ellisfound>
{{cite journal
 |last = Ellis  |first = R.L. |author-link = Robert Leslie Ellis
 |year = 1843
 |title = On the foundations of the theory of probabilities
 |journal = [[Transactions of the Cambridge Philosophical Society]]
 |volume = 8
}}
</ref>
and Ellis (1854),<ref name=ellisfund>
{{cite journal
 |last = Ellis |first = R.L. |author-link = Robert Leslie Ellis
 |year = 1854
 |title = Remarks on the fundamental principles of the theory of probabilities
 |journal = [[Transactions of the Cambridge Philosophical Society]]
 |volume = 9
}}
</ref> [[Antoine Augustin Cournot|Cournot]] (1843),<ref>
{{cite book
 |last = Cournot |first = A.A. |author-link = Antoine Augustin Cournot
 |year = 1843
 |title = Exposition de la théorie des chances et des probabilités
 |publisher = L. Hachette
 |place = Paris, FR
 |url = https://archive.org/details/expositiondelat00courgoog 
 |via = [[Internet Archive]] (archive.org)
}}
</ref>
and [[Jakob Friedrich Fries|Fries]] introduced the frequentist view. [[John Venn|Venn]] (1866, 1876, 1888)<ref name=Venn-1888/> provided a thorough exposition two decades later.
These were further supported by the publications of [[George Boole|Boole]] and [[Joseph Louis François Bertrand|Bertrand]]. By the end of the 19th&nbsp;century the frequentist interpretation was well established and perhaps dominant in the sciences.<ref name=Gig89/> The following generation established the tools of classical inferential statistics (significance testing, hypothesis testing and confidence intervals) all based on frequentist probability.

Alternatively,<ref name=Anders-2004/>
[[Jacob Bernoulli|Bernoulli]]{{efn|
The Swiss mathematician [[Jacob Bernoulli]] of the famous [[Bernoulli family]] lived in a multi-lingual country and himself had regular correspondance and contacts with speakers of German and French, and published in Latin – all of which he spoke fluently. He comfortably and frequently used the three names "Jacob", "James", and "Jacques", depending on the language he was speaking or writing.
}}
understood the concept of frequentist probability and published a critical proof (the [[weak law of large numbers]]) posthumously (Bernoulli, 1713).<ref>{{cite book
 |first = Jakob  |last = Bernoulli
 |year=1713
 |title = Ars Conjectandi: Usum & applicationem praecedentis doctrinae in civilibus, moralibus, & oeconomicis
 |lang=la
 |trans-title = The Art of Conjecture: The use and application of previous experience in civil, moral, and economic topics
}}
</ref>
He is also credited with some appreciation for subjective probability (prior to and without [[Bayes' theorem]]).<ref>
{{cite journal
 | last = Fienberg | first = Stephen E.
 | year = 1992
 | title = A Brief History of Statistics in Three and One-half Chapters: A Review Essay
 | journal = Statistical Science
 | volume = 7 | number = 2 | pages = 208–225
 | doi=10.1214/ss/1177011360| doi-access = free
}}
</ref>{{efn|
Bernoulli provided a classical example of drawing many black and white pebbles from an urn (with replacement). The sample ratio allowed Bernoulli to infer the ratio in the urn, with tighter bounds as the number of samples increased.
:
Historians can interpret the example as classical, frequentist, or subjective probability. David writes, ''"[[Jacob Bernoulli|James]] has definitely started here the controversy on inverse probability&nbsp;..."'' Bernoulli wrote generations before Bayes, LaPlace and Gauss. The controversy continues. — {{harvp|David|1962|pp= 137–138}}<ref name=David-1962/>
}}<ref name=David-1962>
{{cite book
 | last = David | first = F.N.
 | year = 1962
 | title = Games, Gods, & Gambling
 | location = New York, NY
 | publisher = Hafner
 | pages = 137–138
}}
</ref>
[[Carl Friedrich Gauss|Gauss]] and [[Pierre-Simon Laplace|Laplace]] used frequentist (and other) probability in derivations of the least squares method a century later, a generation before Poisson.<ref name=Anders-2004>
{{cite book
 | last = Hald | first = Anders
 | year = 2004
 | title = A history of Parametric Statistical Inference from Bernoulli to Fisher, 1713 to 1935
 | publisher = Anders Hald, Department of Applied Mathematics and Statistics, [[University of Copenhagen]]
 | location = København, DM
 | isbn = 978-87-7834-628-5
 | pages = 1–5
}}
</ref>
[[Pierre-Simon Laplace|Laplace]] considered the probabilities of testimonies, tables of mortality, judgments of tribunals, etc. which are unlikely candidates for classical probability. In this view, Poisson's contribution was his sharp criticism of the alternative "inverse" (subjective, Bayesian) probability interpretation. Any criticism by [[Carl Friedrich Gauss|Gauss]] or [[Pierre-Simon Laplace|Laplace]] was muted and implicit. (However, note that their later derivations of [[least squares]] did not use inverse probability.)

Major contributors to "classical" statistics in the early 20th century included [[Ronald Aylmer Fisher|Fisher]], [[Jerzy Neyman|Neyman]], and [[Egon Pearson|Pearson]]. Fisher contributed to most of statistics and made significance testing the core of experimental science, although he was critical of the frequentist concept of ''"repeated sampling from the same population"'';<ref>
{{cite journal
 |last = Rubin |first = M.
 |year=2020
 |title="Repeated sampling from the same population?" A critique of Neyman and Pearson's responses to Fisher
 |journal=European Journal for Philosophy of Science
 |volume=10 |issue=42 |pages=1–15
 |doi=10.1007/s13194-020-00309-6
 |s2cid=221939887
 |url=https://doi.org/10.1007/s13194-020-00309-6
}}
</ref>
Neyman formulated confidence intervals and contributed heavily to sampling theory; Neyman and Pearson paired in the creation of hypothesis testing. All valued objectivity, so the best interpretation of probability available to them was frequentist.

All were suspicious of "inverse probability" (the available alternative) with prior probabilities chosen by using the principle of indifference. Fisher said, ''"...&nbsp;the theory of inverse probability is founded upon an error, [referring to Bayes' theorem] and must be wholly rejected."''<ref>
{{cite book
 |first = R.A. |last = Fisher  |author-link = Ronald Aylmer Fisher
 |title = Statistical Methods for Research Workers
}}
</ref>
While Neyman was a pure frequentist,<ref name=Neyman-1937>
{{cite journal
 |last = Neyman |first = Jerzy  |author-link = Jerzy Neyman
 |date = 30 August 1937
 |title = Outline of a theory of statistical estimation based on the classical theory of probability
 |journal = [[Philosophical Transactions of the Royal Society of London]] A
 |volume = 236  |issue = 767  |pages = 333–380
 |bibcode = 1937RSPTA.236..333N
 |doi = 10.1098/rsta.1937.0005 |doi-access=free
}}
</ref>{{efn|
[[Jerzy Neyman]]'s derivation of confidence intervals embraced the measure theoretic axioms of probability published by [[Andrey Kolmogorov]] a few years earlier, and referenced the '''subjective probability'' (Bayesian) definitions that [[Sir Harold Jeffreys|Jeffreys]] had published earlier in the decade. Neyman defined ''frequentist probability'' (under the name ''classical'') and stated the need for randomness in the repeated samples or trials. He accepted in principle the possibility of multiple competing theories of probability, while expressing several specific reservations about the existing alternative probability interpretation.<ref name=Neyman-1937/>
}}
Fisher's views of probability were unique: Both Fisher and Neyman had nuanced view of probability. [[Richard von Mises|von Mises]] offered a combination of mathematical and philosophical support for frequentism in the era.<ref>
{{cite book
 |last = von Mises |first = Richard  |author-link = Richard von Mises
 |orig-year = 1939  |year = 1981
 |title = Probability, Statistics, and Truth  |edition = 2nd, rev.
 |lang = de, en
 |publisher = Dover Publications
 |ISBN = 0486242145
 |page = 14 
}}
</ref><ref>
{{cite book
 |first = Donald  |last = Gilles
 |year = 2000
 |chapter=Chapter&nbsp;5 – The frequency theory
 |title = Philosophical Theories of Probability
 |publisher = Psychology Press
 |ISBN = 9780415182751
 |page = 88
}}
</ref>

==Etymology==
According to the ''[[Oxford English Dictionary]]'', the term ''frequentist'' was first used by [[Maurice Kendall|M.G. Kendall]] in 1949, to contrast with [[Bayesian probability|Bayesians]], whom he called ''non-frequentists''.<ref>
{{cite web
 | title = Earliest known uses of some of the words of probability & statistics
 | website = leidenuniv.nl 
 | publisher = [[Leiden University]]
 | place = Leidin, NL
 | url = http://www.leidenuniv.nl/fsw/verduin/stathist/1stword.htm
}}
</ref><ref name=Kendall-1949>
{{cite journal
 |last = Kendall |first = M.G. |author-link = Maurice Kendall
 |year = 1949
 |title = On the Reconciliation of Theories of Probability
 |journal = [[Biometrika]]
 |volume=36  |issue=1-2  |pages=101–116
 |jstor=2332534  |pmid=18132087
 |doi=10.1093/biomet/36.1-2.101
}}
</ref>
[[Maurice Kendall|Kendall]] observed
:3. ... we may broadly distinguish two main attitudes. One takes probability as 'a degree of rational belief', or some similar idea...the second defines probability in terms of frequencies of occurrence of events, or by relative proportions in 'populations' or 'collectives';<ref name=Kendall-1949/>{{rp|style=ama|p= 101}}
: ...
:12. It might be thought that the differences between the frequentists and the non-frequentists (if I may call them such) are largely due to the differences of the domains which they purport to cover.<ref name=Kendall-1949/>{{rp|style=ama|p= 104}}
: ...
:''I assert that this is not so'' ... The essential distinction between the frequentists and the non-frequentists is, I think, that the former, in an effort to avoid anything savouring of matters of opinion, seek to define probability in terms of the objective properties of a population, real or hypothetical, whereas the latter do not. [emphasis in original]

"The Frequency Theory of Probability" was used a generation earlier as a chapter title in Keynes (1921).<ref name=keynesVIII/>

The historical sequence:
# Probability concepts were introduced and much of the mathematics of probability derived (prior to the 20th&nbsp;century)
# classical statistical inference methods were developed
# the mathematical foundations of probability were solidified and current terminology was introduced (all in the 20th century).
The primary historical sources in probability and statistics did not use the current terminology of ''classical'', ''subjective'' (Bayesian), and ''frequentist'' probability.

==Alternative views==
{{Main|Probability interpretations}}
[[Probability theory]] is a branch of mathematics. While its roots reach centuries into the past, it reached maturity with the axioms of [[Andrey Kolmogorov]] in 1933. The theory focuses on the valid operations on probability values rather than on the initial assignment of values; the mathematics is largely independent of any interpretation of probability.

Applications and interpretations of [[probability]] are considered by philosophy, the sciences and statistics. All are interested in the extraction of knowledge from observations—[[inductive reasoning]]. There are a variety of competing interpretations;<ref name=SEPIP>
{{cite encyclopedia
 |last = Hájek  |first = Alan
 |date = 21 October 2002
 |title = Interpretations of probability
 |editor-first = Edward N. |editor-last = Zalta
 |encyclopedia = The Stanford Encyclopedia of Philosophy
 |url = http://plato.stanford.edu/archives/win2012/entries/probability-interpret/
 |via = plato.stanford.edu
}}
</ref>
All have problems. The frequentist interpretation does resolve difficulties with the classical interpretation, such as any problem where the natural symmetry of outcomes is not known. It does not address other issues, such as the [[dutch book]].

* [[Classical definition of probability|Classical probability]] assigns probabilities based on physical idealized symmetry (dice, coins, cards). The classical definition is at risk of circularity: Probabilities are defined by assuming equality of probabilities.<ref name=Ash>
{{cite book
 |last = Ash  |first = Robert B.
 |year = 1970
 |title = Basic Probability Theory
 |publisher = Wiley
 |location = New York, NY
 |pages = 1–2
}}
</ref> In the absence of symmetry the utility of the definition is limited.

* [[Bayesian probability|Subjective (Bayesian) probability]] (a family of competing interpretations) considers degrees of belief: All practical "subjective" probability interpretations are so constrained to rationality as to avoid most subjectivity. Real subjectivity is repellent to some definitions of science which strive for results independent of the observer and analyst.{{citation needed|date=October 2019}} Other applications of Bayesianism in science (e.g. logical Bayesianism) embrace the inherent subjectivity of many scientific studies and objects and use Bayesian reasoning to place boundaries and context on the influence of [[Subjectivity#Sociology|subjectivities]] on all analysis.<ref>
{{cite journal
 |last1 = Fairfield  |first1 = Tasha
 |last2 = Charman    |first2 = Andrew E.
 |date = 15 May 2017
 |title = Explicit Bayesian analysis for process tracing: Guidelines, opportunities, and caveats
 |journal = [[Political Analysis (journal)|Political Analysis]]
 |volume = 25  |issue = 3  |pages = 363–380
 |doi = 10.1017/pan.2017.14  |s2cid = 8862619
 |url = http://eprints.lse.ac.uk/69203/
}}
</ref>  The historical roots of this concept extended to such non-numeric applications as legal evidence.

* [[Propensity probability]] views probability as a causative phenomenon rather than a purely descriptive or subjective one.<ref name=SEPIP/>

== Footnotes ==
{{notelist}}

== Citations ==
{{reflist|25em|refs=

<ref name=Venn-1888>
{{cite book
 |last = Venn |first = John  |author-link = John Venn
 |year = 1888 |orig-year = 1866, 1876
 |title = The Logic of Chance |edition = 3rd
 |quote = An essay on the foundations and province of the theory of probability, with especial reference to its logical bearings and its application to moral and social science, and to statistics.
 |publisher = Macmillan & Co.
 |place = London, UK
 |url = https://archive.org/details/logicofchance029416mbp
 |via = [[Internet Archive]] (archive.org
}}
</ref>

}}

==References==
{{refbegin|colwidth=25em|small=yes}}

* {{cite book
 |first = P.W. |last = Bridgman  |author-link = 
 |year = 1927
 |title = The Logic of Modern Physics
}}

* {{cite book
 |first = Alonzo |last = Church  |author-link = 
 |year = 1940
 |title = The Concept of a Random Sequence
}}

* {{cite book
 |first = Harald |last = Cramér  |author-link = Harald Cramér 
 |year = 1946
 |title = Mathematical Methods of Statistics
}}

* {{cite book
 |first = William |last = Feller  |author-link = 
 |year = 1957
 |title = An Introduction to Probability Theory and its Applications
}}

* {{cite book
 |first = P. |last = Martin-Löf  |author-link = 
 |year = 1966
 |title = On the Concept of a Random Sequence
}}

* {{cite book
 |first = Richard  |last = von Mises  |author-link = Richard von Mises
 |year = 1939  |lang = en |orig-year = 1928 (in German)
 |title = Probability, Statistics, and Truth
}}

* {{cite book
 |first = Jerzy  |last = Neyman  |author-link = Jerzy Neyman
 |year = 1950
 |title = First Course in Probability and Statistics
}}

* {{cite book
 |first = Hans |last = Reichenbach  |author-link = 
 |title = The Theory of Probability
 |year = 1949 |lang = en |orig-year = 1935 (in German)
}}

* {{cite book
 |first = Bertrand |last = Russell  |author-link = Bertrand Russell
 |title = Human Knowledge
 |year = 1948
}}

* {{cite journal
 |last = Friedman |first = C.  |author-link = 
 |year = 1999
 |title = The frequency interpretation in probability
 |journal = [[Advances in Applied Mathematics]]
 |volume = 23  |issue = 3  |pages = 234–254
 |doi = 10.1006/aama.1999.0653   |doi-access = free
}} {{cite web
 |title = alt. source
 |website = utexas.edu/~friedman
 |place = Austin, TX
 |publisher = [[University of Texas]]
 |url = http://www.ma.utexas.edu/~friedman/freq.ps
 |format=PS
}}

{{refend}}

{{statistics}}

{{DEFAULTSORT:Frequency Probability}}
[[Category:Probability interpretations]]