Editing Power (statistics)

{{Short description|Term in statistical hypothesis testing}}
In [[frequentist]] [[statistics]], '''power''' is the probability of detecting a given effect (if that effect actually exists) using a given test in a given context. In typical use, it is a function of the specific test that is used (including the choice of test statistic and [[significance level]]), the [[sample size]] (more data tends to provide more power), and the [[effect size]] (effects or correlations that are large relative to the variability of the data tend to provide more power).

More formally, in the case of a simple [[hypothesis test]] with two hypotheses, the '''power of the test''' is the probability that the test correctly rejects the [[null hypothesis]] (<math>H_0</math>) when the [[alternative hypothesis]] (<math>H_1</math>) is true. It is commonly denoted by <math>1-\beta</math>, where <math>\beta</math> is the probability of making a [[Type I and type II errors#Type II error|type&nbsp;II error]] (a [[false negative]]) [[conditional probability|conditional]] on there being a true effect or association.

==Background==
{{main|Statistical hypothesis test}}
[[Statistical test]]ing uses data from [[Sampling (statistics)|sample]]s to assess, or make [[statistical inference|inferences]] about, a [[statistical population]]. For example, we may measure the yields of samples of two varieties of a crop, and use a two sample test to assess whether the [[mean values]] of this yield differs between varieties.

Under a frequentist hypothesis testing framework, this is done by calculating a test statistic (such as a [[t-statistic]]) for the dataset, which has a known theoretical [[probability distribution]] if there is no difference (the so called null hypothesis). If the actual value calculated on the sample is sufficiently unlikely to arise under the null hypothesis, we say we identified a statistically significant effect.

The threshold for significance can be set small to ensure there is little chance of falsely detecting a non-existent effect. However, failing to identify a significant effect does not imply there was none. If we insist on being careful to avoid false positives, we may create false negatives instead. It may simply be too much to expect that we will be able to find satisfactorily strong evidence of a very subtle difference even if it exists. Statistical power is an attempt to quantify this issue.

In the case of the comparison of the two crop varieties, it enables us to answer questions like:
* Is there a big danger of two very different varieties producing samples that just happen to look indistinguishable by pure chance?
* How much effort do we need to put into this comparison to avoid that danger?
* How different do these varieties need to be before we can expect to notice a difference?

==Description==
{{see also|Type I and type II errors}}
[[File:PowerOfTest.png|thumb|Illustration of the power of a statistical test, for a two sided test, through the probability distribution of the test statistic under the null and alternative hypothesis. ''α'' is shown as the <span style="color:blue">blue area</span>, the probability of rejection under null, while the <span style="color:red">red area</span> shows power, 1 − ''β'', the probability of correctly rejecting under the alternative.]]

Suppose we are conducting a hypothesis test. We define two hypotheses <math>H_0</math> the null hypothesis, and <math>H_1</math> the alternative hypothesis. If we design the test such that ''α'' is the significance level - being the probability of rejecting <math>H_0</math> when <math>H_0</math> is in fact true, then the power of the test is 1 - ''β'' where ''β'' is the probability of failing to reject <math>H_0</math> when the alternative <math>H_1</math> is true.

{| class="wikitable"
!   !! Probability to reject <math>H_0</math> !! Probability to not reject <math>H_0</math>
|-
| If <math>H_0</math> is True  || α || 1-α
|-
| If <math>H_1</math> is True || 1-β (power) || β
|}

To make this more concrete, a typical statistical test would be based on a [[test statistic]] ''t'' calculated from the sampled data, which has a particular [[probability distribution]] under <math>H_0</math>. A desired [[significance level]] ''α'' would then define a corresponding "rejection region" (bounded by certain "critical values"), a set of values ''t'' is unlikely to take if <math>H_0</math> was correct. If we reject <math>H_0</math> in favor of <math>H_1</math> only when the sample ''t'' takes those values, we would be able to keep the probability of falsely rejecting <math>H_0</math> within our desired significance level. At the same time, if <math>H_1</math> defines its own probability distribution for ''t'' (the difference between the two distributions being a function of the effect size), the power of the test would be the probability, under <math>H_1</math>, that the sample ''t'' falls into our defined rejection region and causes <math>H_0</math> to be correctly rejected.

Statistical power is one minus the type&nbsp;II error probability and is also the [[Sensitivity and specificity|sensitivity]] of the hypothesis testing procedure to detect a true effect. There is usually a trade-off between demanding more stringent tests (and so, smaller rejection regions) and trying to have a high probability of rejecting the null under the alternative hypothesis. Statistical power may also be extended to the case where [[multiple testing|multiple hypotheses]] are being tested based on an experiment or survey. It is thus also common to refer to the '''power of a study''', evaluating a scientific project in terms of its ability to answer the [[research question]]s they are seeking to answer.

== Applications ==
The main application of statistical power is "power analysis", a calculation of power usually done before an experiment is conducted using data from [[pilot studies]] or a literature review. Power analyses can be used to calculate the minimum [[sample size]] required so that one can be reasonably likely to detect an [[effect size|effect of a given size]] (in other words, producing an acceptable level of power). For example: "How many times do I need to toss a coin to conclude it is rigged by a certain amount?"<ref>{{cite web |url=https://www.statisticsdonewrong.com/power.html |title=Statistical power and underpowered statistics — Statistics Done Wrong |website=www.statisticsdonewrong.com |access-date=30 September 2019}}</ref> If resources and thus sample sizes are fixed, power analyses can also be used to calculate the minimum effect size that is likely to be detected.

Funding agencies, ethics boards and research review panels frequently request that a researcher perform a power analysis. An underpowered study is likely be inconclusive, failing to allow one to choose between hypotheses at the desired significance level, while an overpowered study will spend great expense on being able to report significant effects even if they are tiny and so practically meaningless. If a large number of underpowered studies are done and [[publication bias|statistically significant results published]], published findings are more likely false positives than true results, contributing to a [[replication crisis]]. However, excessive demands for power could be connected to wasted resources and ethical problems, for example the use of a large number of animal test subjects when a smaller number would have been sufficient. It could also induce researchers trying to seek funding to overstate their expected effect sizes, or avoid looking for more subtle interaction effects that cannot be easily detected.<ref>{{cite journal|title=Finding the right power balance: Better study design and collaboration can reduce dependence on statistical power|first1=Shinichi|last1=Nakagawa|year=2024|journal=PLOS Biology|first2=Malgorzata|last2=Lagisz|first3=Yefeng |last3=Yang|first4=Szymon M. |last4=Drobniak|volume=22 |issue=1 |pages=e3002423 |doi=10.1371/journal.pbio.3002423|doi-access=free |pmid=38190355 |pmc=10773938 }}</ref>

Power analysis is primarily a [[frequentist statistics]] tool. In [[Bayesian statistics]], hypothesis testing of the type used in classical power analysis is not done. In the Bayesian framework, one updates his or her prior beliefs using the data obtained in a given study. In principle, a study that would be deemed underpowered from the perspective of hypothesis testing could still be used in such an updating process. However, power remains a useful measure of how much a given experiment size can be expected to refine one's beliefs. A study with low power is unlikely to lead to a large change in beliefs.

In addition, the concept of power is used to make comparisons between different statistical testing procedures: for example, between a [[Parametric statistics|parametric test]] and a [[nonparametric test]] of the same hypothesis. Tests may have the same [[Size (statistics)|size]], and hence the same false positive rates, but different ability to detect true effects. Consideration of their theoretical power proprieties is a key reason for the common use of [[likelihood ratio test]]s.

== Rule of thumb for t-test ==
Lehr's<ref>{{citation|surname1=Robert Lehr|periodical=Statistics in Medicine|title=SixteenS-squared overD-squared: A relation for crude sample size estimates|volume=11|issue=8|at=pp.&nbsp;1099–1102|issn=0277-6715|date=1992|language=German|doi=10.1002/sim.4780110811
|pmid=1496197 }}</ref><ref>{{Cite book|last=van Belle|first=Gerald|url=http://dx.doi.org/10.1002/9780470377963|title=Statistical Rules of Thumb, Second Edition|date=2008-08-18|publisher=John Wiley & Sons, Inc.|isbn=978-0-470-37796-3|series=Wiley Series in Probability and Statistics|location=Hoboken, NJ, USA|doi=10.1002/9780470377963 }}</ref> (rough) [[rule of thumb]] says that the sample size <math>n</math> (for each group) for the common case of a two-sided [[two-sample t-test]] with power 80% (<math>\beta = 0.2</math>) and [[significance level]] <math>\alpha = 0.05</math> should be:
<math display="block">n\approx16\frac{s^2}{d^2},</math>
where <math>s^2</math> is an estimate of the population variance and <math>d=\mu_1-\mu_2</math> the to-be-detected difference in the mean values of both samples. This expression can be rearranged, implying for example that 80% power is obtained when looking for a difference in means that exceeds about 4 times the group-wise [[standard error of the mean]].

For a [[one sample t-test]] 16 is to be replaced with 8. Other values provide an appropriate approximation when the desired power or significance level are different.<ref>Sample Size Estimation in Clinical Research From Randomized Controlled Trials to Observational Studies, 2020, doi: 10.1016/j.chest.2020.03.010, Xiaofeng Wang, PhD; and Xinge Ji, MS [https://journal.chestnet.org/action/showPdf?pii=S0012-3692%2820%2930458-X pdf]</ref>

However, a full power analysis should always be performed to confirm and refine this estimate.

==Factors influencing power==
[[File:Sample Sizes Effect on Power.png|thumb|An example of the relationship between sample size and power levels. Higher power requires larger sample sizes]]
Statistical power may depend on a number of factors. Some factors may be particular to a specific testing situation, but in normal use, power depends on the following three aspects that can be potentially controlled by the practitioner:

* the test itself and the [[statistical significance]] criterion used
* the magnitude of the effect of interest
* the [[sample size|size]] and [[Statistical dispersion|variability]] of the sample used to detect the effect

For a given test, the '''significance criterion''' determines the desired degree of rigor, specifying how unlikely it is for the null hypothesis of no effect to be rejected if it is in fact true. The most commonly used threshold is a probability of rejection of 0.05, though smaller values like 0.01 or 0.001 are sometimes used. This threshold then implies that the observation must be at least that unlikely (perhaps by suggesting a sufficiently large estimate of difference) to be considered strong enough evidence against the null. Picking a smaller value to tighten the threshold, so as to reduce the chance of a false positive, would also reduce power, increase the chance of a false negative. Some statistical tests will [[uniformly most powerful test|inherently produce better power]], albeit often at the cost of requiring stronger assumptions.

The '''magnitude of the effect''' of interest defines what is being looked for by the test. It can be the expected [[effect size]] if it exists, as a scientific [[hypothesis]] that the researcher has arrived at and wishes to test. Alternatively, in a more practical context it could be determined by the size the effect must be to be useful, for example that which is required to be [[clinically significant]]. An effect size can be a direct value of the quantity of interest (for example, a difference in mean of a particular size), or it can be a standardized measure that also accounts for the variability in the population (such as a difference in means expressed as a multiple of the standard deviation). If the researcher is looking for a larger effect, then it should be easier to find with a given experimental or analytic setup, and so power is higher.

The nature of the '''sample''' underlies the information being used in the test. This will usually involve the sample size, and the sample variability, if that is not implicit in the definition of the effect size. More broadly, the precision with which the data are measured can also be an important factor (such as the [[Reliability (statistics)|statistical reliability]]), as well as the [[design of experiments|design]] of an experiment or observational study. Ultimately, these factors lead to an expected amount of [[sampling error]]. A smaller sampling error could be obtained by larger sample sizes from a less variability population, from more accurate measurements, or from more efficient experimental designs (for example, with the appropriate use of [[Blocking (statistics)|blocking]]), and such smaller errors would lead to improved power, albeit usually at a cost in resources. How increased sample size translates to higher power is a measure of the [[efficiency (statistics)|efficiency]] of the test&nbsp;– for example, the sample size required for a given power.<ref name=Everitt2002>{{cite book |title=The Cambridge Dictionary of Statistics |last=Everitt |first=Brian S. |year=2002 |publisher=Cambridge University Press |isbn=0-521-81099-X |page=321}}</ref>

==Discussion==
The statistical power of a hypothesis test has an impact on the interpretation of its results. Not finding a result with a more powerful study is stronger evidence against the effect existing than the same finding with a less powerful study. However, this is not completely conclusive. The effect may exist, but be smaller than what was looked for, meaning the study is in fact underpowered and the sample is thus unable to distinguish it from random chance.<ref>{{cite book |title=The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and the Interpretation of Research Results |last=Ellis |first=Paul |year=2010 |isbn=978-0521142465 |publisher=Cambridge University Press |page=52}}</ref> Many [[clinical trial]]s, for instance, have low statistical power to detect differences in [[adverse effect]]s of treatments, since such effects may only affect a few patients, even if this difference can be [[clinical significance|important]].<ref>{{cite journal |last1=Tsang |first1=R. |last2=Colley |first2=L. |last3=Lynd |first3=L.D. |doi=10.1016/j.jclinepi.2008.08.005 |title=Inadequate statistical power to detect clinically significant differences in adverse event rates in randomized controlled trials |journal=Journal of Clinical Epidemiology |volume=62 |issue=6 |pages=609–616 |year=2009 |pmid=19013761}}</ref> Conclusions about the [[Posterior probability|probability of actual presence]] of an effect also should consider more things than a single test, especially as real world power is rarely close to 1.

Indeed, although there are no formal standards for power, many researchers and funding bodies assess power using 0.80 (or 80%) as a standard for adequacy. This convention implies a four-to-one trade off between {{mvar|β}}-risk and {{mvar|α}}-risk, as the probability of a type&nbsp;II error {{mvar|β}} is set as 1 - 0.8 = 0.2, while α, the probability of a type&nbsp;I error, is commonly set at 0.05. Some applications require much higher levels of power. [[Medical test]]s may be designed to minimise the number of false negatives (type&nbsp;II errors) produced by loosening the threshold of significance, raising the risk of obtaining a false positive (a type&nbsp;I error). The rationale is that it is better to tell a healthy patient "we may have found something—let's test further," than to tell a diseased patient "all is well."<ref>{{cite book |author=Ellis, Paul D. |title=The Essential Guide to Effect Sizes: An Introduction to Statistical Power, Meta-Analysis and the Interpretation of Research Results |publisher=Cambridge University Press |location=United Kingdom |year=2010|page = 56}}</ref>

Power analysis focuses on the correct rejection of a null hypothesis. Alternative concerns may however motivate an experiment, and so lead to different needs for sample size. In many contexts, the issue is less about deciding between hypotheses but rather with getting an [[estimation theory|estimate]] of the population effect size of sufficient accuracy. For example, a careful power analysis can tell you that 55 pairs of normally distributed samples with a [[Pearson product-moment correlation coefficient|correlation]] of 0.5 will be sufficient to grant 80% power in rejecting a null that the correlation is no more than 0.2 (using a one-sided test, {{mvar|α}}&nbsp;=&nbsp;0.05). But the typical 95% [[confidence interval]] with this sample would be around [0.27, 0.67]. An alternative, albeit related analysis would be required if we wish to be able to measure correlation to an accuracy of +/- 0.1, implying a different (in this case, larger) sample size. Alternatively, multiple under-powered studies can still be useful, if appropriately combined through a [[meta-analysis]].

Many statistical analyses involve the estimation of several unknown quantities. In simple cases, all but one of these quantities are [[nuisance parameter]]s. In this setting, the only relevant power pertains to the single quantity that will undergo formal statistical inference. In some settings, particularly if the goals are more "exploratory", there may be a number of quantities of interest in the analysis. For example, in a multiple [[regression analysis]] we may include several covariates of potential interest. In situations such as this where several hypotheses are under consideration, it is common that the powers associated with the different hypotheses differ. For instance, in multiple regression analysis, the power for detecting an effect of a given size is related to the variance of the covariate. Since different covariates will have different variances, their powers will differ as well.

Additional complications arise when we consider these [[multiple comparisons|multiple hypotheses]] together. For example, if we consider a false positive to be making an erroneous null rejection on any one of these hypotheses, our likelihood of this [[Family-wise error rate|"family-wise error"]] will be inflated if appropriate measures are not taken. Such measures typically involve applying a higher threshold of stringency to reject a hypothesis (such as with the [[Bonferroni method]]), and so would reduce power. Alternatively, there may be different notions of power connected with how the different hypotheses are considered. "Complete power" demands that all true effects are detected across all of the hypotheses, which is a much stronger requirement than the "minimal power" of being able to find at least one true effect, a type of power that might increase with an increasing number of hypotheses.<ref>{{cite web|title= Estimating Statistical Power When Using Multiple Testing Procedures|url=https://www.mdrc.org/work/publications/estimating-statistical-power-when-using-multiple-testing-procedures|website=mdrc.org|date=November 2017 }}</ref>

==''A priori'' vs. ''post hoc'' analysis==
{{further|Post hoc analysis}}
Power analysis can either be done before (''a priori'' or prospective power analysis) or after (''post hoc'' or retrospective power analysis) data are collected. ''A priori'' power analysis is conducted prior to the research study, and is typically used in [[estimating sample sizes|estimating sufficient sample sizes]] to achieve adequate power. ''Post-hoc'' analysis of "observed power" is conducted after a study has been completed, and uses the obtained sample size and effect size to determine what the power was in the study, assuming the effect size in the sample is equal to the effect size in the population. Whereas the utility of prospective power analysis in experimental design is universally accepted, post hoc power analysis is controversial. Many statisticians have argued that post-hoc power calculations are misleading and essentially meaningless.<ref name="HH1">{{cite journal |last1=Hoenig |last2=Heisey |year=2001 |title=The Abuse of Power |journal=[[The American Statistician]] |volume=55 |issue=1 |pages=19–24 |doi=10.1198/000313001300339897}}</ref><ref>{{cite journal |author=Thomas, L. |year=1997 |url=http://eprints.st-andrews.ac.uk/archive/00000417/01/ThomasCB1997.pdf |title=Retrospective power analysis |journal=[[Conservation Biology (journal)|Conservation Biology]] |volume=11 |issue=1 |pages=276–280|doi=10.1046/j.1523-1739.1997.96102.x |bibcode=1997ConBi..11..276T |hdl=10023/679 }}</ref>

==Example==
The following is an example that shows how to compute power for a randomized experiment: Suppose the goal of an experiment is to study the effect of a treatment on some quantity, and so we shall compare research subjects by measuring the quantity before and after the treatment, analyzing the data using a one-sided [[Paired difference test|paired]] [[t-test]], with a significance level threshold of 0.05. We are interested in being able to detect a positive change of size <math>\theta > 0</math>.

We first set up the problem according to our test. Let <math>A_i</math> and <math>B_i</math> denote the pre-treatment and post-treatment measures on subject <math>i</math>, respectively. The possible effect of the treatment should be visible in the differences <math>D_i = B_i-A_i,</math> which are assumed to be independent and identically [[normal distribution|Normal]] in distribution, with unknown mean value <math>\mu_D</math> and variance <math>\sigma_D^2</math>.

Here, it is natural to choose our null hypothesis to be that the expected mean difference is zero, i.e. <math>H_0: \mu_D =\mu_0= 0.</math> For our one-sided test, the alternative hypothesis would be that there is a positive effect, corresponding to <math>H_1: \mu_D = \theta > 0.</math> The [[test statistic]] in this case is defined as:

<math display="block">T_n=\frac{\bar{D}_n-\mu_0 }{\hat{\sigma}_D/\sqrt{n}} =\frac{\bar{D}_n-0}{\hat{\sigma}_D/\sqrt{n}},</math>

where <math>\mu_0</math> is the mean under the null so we substitute in 0, {{mvar|n}} is the sample size (number of subjects), <math>\bar{D}_n</math> is the [[sample mean]] of the difference

<math display="block">\bar{D}_n=\frac{1}{n}\sum_{i=1}^n D_i,</math>

and <math>\hat{\sigma}_D</math> is the sample [[standard deviation]] of the difference.

=== Analytic solution ===

We can proceed according to our knowledge of statistical theory, though in practice for a standard case like this software will exist to compute more accurate answers.

Thanks to t-test theory, we know this test statistic under the null hypothesis follows a [[Student t-distribution]] with <math>n-1</math> degrees of freedom. If we wish to reject the null at significance level <math>\alpha = 0.05\,</math>, we must find the [[Critical value (statistics)|critical value]] <math>t_{\alpha}</math> such that the probability of <math>T_n > t_{\alpha}</math> under the null is equal to <math>\alpha</math>. If {{mvar|n}} is large, the t-distribution converges to the standard normal distribution (thus no longer involving {{mvar|n}}) and so through use of the [[Normal_distribution#Quantile_function|corresponding]] [[quantile function]] <math>\Phi^{-1}</math>, we obtain that the null should be rejected if

<math display="block">T_n > t_{\alpha} \approx \Phi^{-1}(0.95) \approx 1.64\,.</math>

Now suppose that the alternative hypothesis <math>H_1</math> is true so <math>\mu_D = \theta</math>. Then, writing the power as a function of the effect size, <math>B(\theta)</math>, we find the probability of <math>T_n</math> being above <math>t_{\alpha}</math> under <math>H_1</math>.

<math display="block">\begin{align}
B(\theta) &\approx \Pr \left( T_n > 1.64 ~\big|~ \mu_D = \theta \right) \\
&= \Pr \left( \frac{\bar{D}_n-0}{\hat{\sigma}_D/\sqrt{n} } > 1.64 ~\Big|~ \mu_D = \theta \right) \\
&= 1- \Pr \left( \frac{\bar{D}_n-0}{\hat{\sigma}_D/\sqrt{n} } < 1.64 ~\Big|~ \mu_D = \theta \right) \\
&= 1 - \Pr\left( \frac{\bar{D}_n-\theta}{\hat{\sigma}_D/\sqrt{n}} < 1.64 - \frac \theta {\hat{\sigma}_D/\sqrt{n}} ~\Big|~ \mu_D=\theta \right)\\
\end{align}</math>

<math>\frac{\bar{D}_n - \theta}{\hat{\sigma}_D/\sqrt{n}}</math> again follows a student-t distribution under <math>H_1</math>, converging on to a standard [[normal distribution]] for large {{mvar|n}}. The estimated <math>{\hat{\sigma}}_D</math> will also converge on to its population value <math>\sigma_D</math> Thus power can be approximated as

<math display="block">B(\theta) \approx 1 - \Phi \left( 1.64 - \frac{\theta}{\sigma_D/\sqrt{n}} \right). </math>

According to this formula, the power increases with the values of the effect size <math>\theta</math> and the sample size {{mvar|n}}, and reduces with increasing variability <math>\sigma_D</math>. In the trivial case of zero effect size, power is at a minimum ([[infimum]]) and equal to the significance level of the test <math>\alpha\,,</math> in this example 0.05. For finite sample sizes and non-zero variability, it is the case here, as is typical, that power cannot be made equal to 1 except in the trivial case where <math>\alpha = 1</math> so the null is ''always'' rejected.

We can invert <math>B</math> to obtain required sample sizes:

<math display="block">\sqrt{n} > \frac{\sigma_D}{\theta}\left( 1.64- \Phi^{-1} \left( 1- B(\theta)\right) \right).</math>

Suppose <math>\theta = 1</math> and we believe <math>\sigma_D</math> is around 2, say, then we require for a power of <math> B(\theta) = 0.8</math>, a sample size

<math display="block">n > 4 \left( 1.64- \Phi^{-1} \left( 1- 0.8\right) \right)^2 \approx 4 \left( 1.64+0.84\right)^2 \approx 24.6 .</math>

=== Simulation solution ===

Alternatively we can use a [[Monte Carlo simulation]] method that works more generally.<ref>{{cite conference|url=https://support.sas.com/resources/papers/proceedings/proceedings/sugi24/Posters/p236-24.pdf|last=Graebner|first=Robert W.|title=Study design with SAS: Estimating power with Monte Carlo methods|year=1999|conference=SUGI 24}}</ref> Once again, we return to the assumption of the distribution of <math>D_n</math> and the definition of <math>T_n</math>. Suppose we have fixed values of the sample size, variability and effect size, and wish to compute power. We can adopt this process:

1. Generate a large number of sets of <math>D_n</math> according to the null hypothesis, <math>N(0, \sigma_D)</math>

2. Compute the resulting test statistic <math>T_n</math> for each set.

3. Compute the <math>(1-\alpha)</math>th quantile of the simulated <math>T_n</math> and use that as an estimate of <math>t_\alpha</math>.

4. Now generate a large number of sets of <math>D_n</math> according to the alternative hypothesis, <math>N(\theta, \sigma_D)</math>, and compute the corresponding test statistics again.

5. Look at the proportion of these simulated alternative <math>T_n</math> that are above the <math>t_\alpha</math> calculated in step 3 and so are rejected. This is the power.

This can be done with a variety of software packages. Using this methodology with the values before, setting the sample size to 25 leads to an estimated power of around 0.78. The small discrepancy with the previous section is due mainly to inaccuracies with the normal approximation.

== Extension ==
=== Bayesian power ===
In the [[frequentist]] setting, parameters are assumed to have a specific value which is unlikely to be true. This issue can be addressed by assuming the parameter has a distribution. The resulting power is sometimes referred to as Bayesian power which is commonly used in [[clinical trial]] design.

=== Predictive probability of success ===
Both [[frequentist]] power and Bayesian power use statistical significance as the success criterion. However, statistical significance is often not enough to define success. To address this issue, the power concept can be extended to the concept of [[predictive probability of success]] (PPOS). The success criterion for PPOS is not restricted to statistical significance and is commonly used in [[clinical trial]] designs.

==Software for power and sample size calculations==
Numerous free and/or open source programs are available for performing power and sample size calculations. These include	 
* [[G*Power]] (https://www.gpower.hhu.de/)
* WebPower Free online statistical power analysis (https://webpower.psychstat.org)
* Free and open source online calculators (https://powerandsamplesize.com)
* PowerUp! provides convenient excel-based functions to determine minimum detectable effect size and minimum required sample size for various experimental and quasi-experimental designs.
* PowerUpR is R package version of PowerUp! and additionally includes functions to determine sample size for various multilevel randomized experiments with or without budgetary constraints.
* R package pwr
* R package WebPower
* Python package statsmodels (https://www.statsmodels.org/)

==See also==
{{Portal|Mathematics}}
* {{Annotated link|Positive and negative predictive values}}
* {{Annotated link|Effect size}}
* {{Annotated link|Efficiency (statistics)|Efficiency}}
* {{Annotated link|Neyman–Pearson lemma}}
* {{Annotated link|Sample size}}
* {{Annotated link|Uniformly most powerful test}}

==References==
{{reflist|25em}}

==Sources==
*{{cite book |author-link=Jacob Cohen (statistician) |last=Cohen |first=J. |title=Statistical Power Analysis for the Behavioral Sciences |edition=2nd |year=1988 |publisher=Lawrence Erlbaum Associates |isbn=0-8058-0283-5 }}
*{{cite book |last=Aberson |first=C.L. |title=Applied Power Analysis for the Behavioral Science |year=2010 |publisher=Routledge |isbn=978-1-84872-835-6 }}

==External links==
* {{YouTube|UFhJefdVCjE|StatQuest: P-value pitfalls and power calculations}}

{{Statistics|inference}}

{{DEFAULTSORT:Statistical Power}}
[[Category:Statistical hypothesis testing]]