Editing Statistical significance (section)

== Challenges ==
{{See also|Misuse of p-values}}

=== Overuse in some journals ===
Starting in the 2010s, some journals began questioning whether significance testing, and particularly using a threshold of {{math|''α''}}=5%, was being relied on too heavily as the primary measure of validity of a hypothesis.<ref>{{Cite web |url=http://www.education.leeds.ac.uk/events/2015/cssme-seminar-series-the-argument-over-p-values-and-the-null-hypothesis-significance-testing-nhst-paradigm |title=CSSME Seminar Series: The argument over ''p''-values and the Null Hypothesis Significance Testing (NHST) paradigm  |publisher=School of Education, University of Leeds |website=www.education.leeds.ac.uk |access-date=2016-12-01}}</ref> Some journals encouraged authors to do more detailed analysis than just a statistical significance test. In social psychology, the journal ''[[Basic and Applied Social Psychology]]'' banned the use of significance testing altogether from papers it published,<ref>{{cite web | title=Psychology Journal Bans Significance Testing | author=Novella, Steven | date=February 25, 2015 | publisher=Science-Based Medicine | url=https://www.sciencebasedmedicine.org/psychology-journal-bans-significance-testing}}</ref> requiring authors to use other measures to evaluate hypotheses and impact.<ref>{{Cite journal|last=Woolston|first=Chris|date=2015-03-05|title=Psychology journal bans P values|journal=Nature|volume=519|issue=7541|pages=9|doi=10.1038/519009f|bibcode=2015Natur.519....9W |doi-access=free}}</ref><ref>{{Cite news|url=https://www.sciencenews.org/blog/context/p-value-ban-small-step-journal-giant-leap-science|title=P value ban: small step for a journal, giant leap for science|last=Siegfried|first=Tom|date=2015-03-17|newspaper=Science News|access-date=2016-12-01}}</ref>

Other editors, commenting on this ban have noted: "Banning the reporting of ''p''-values, as Basic and Applied Social Psychology recently did, is not going to solve the problem because it is merely treating a symptom of the problem. There is nothing wrong with hypothesis testing and ''p''-values per se as long as authors, reviewers, and action editors use them correctly."<ref>{{Cite journal|last=Antonakis|first=John|date=February 2017|title=On doing better science: From thrill of discovery to policy implications|journal=The Leadership Quarterly|volume=28|issue=1|pages=5–21|doi=10.1016/j.leaqua.2017.01.006|url=https://serval.unil.ch/resource/serval:BIB_8EF01CB80A64.P001/REF.pdf}}</ref> Some statisticians prefer to use alternative measures of evidence, such as [[likelihood ratio]]s or [[Bayes factor]]s.<ref name="Wasserstein 129–133"/> Using [[Bayesian statistics]] can avoid confidence levels, but also requires making additional assumptions,<ref name="Wasserstein 129–133">{{Cite journal|last1=Wasserstein|first1=Ronald L.|last2=Lazar|first2=Nicole A.|date=2016-04-02|title=The ASA's Statement on p-Values: Context, Process, and Purpose|journal= [[The American Statistician]]|volume=70|issue=2|pages=129–133| doi=10.1080/00031305.2016.1154108|url=http://revistas.ucm.es/index.php/TEKN/article/view/57194|doi-access=free}}</ref> and may not necessarily improve practice regarding statistical testing.<ref>{{Cite journal|last=García-Pérez|first=Miguel A.|date=2016-10-05|title=Thou Shalt Not Bear False Witness Against Null Hypothesis Significance Testing|journal=Educational and Psychological Measurement|volume=77|issue=4|language=en|pages=631–662|doi=10.1177/0013164416668232|pmid=30034024|pmc=5991793|issn=0013-1644}}</ref>

The widespread abuse of statistical significance represents an important topic of research in [[metascience (research)|metascience]].<ref>{{cite journal |last1=Ioannidis |first1=John P. A. |last2=Ware |first2=Jennifer J. |last3=Wagenmakers |first3=Eric-Jan |last4=Simonsohn |first4=Uri |last5=Chambers |first5=Christopher D. |last6=Button |first6=Katherine S. |last7=Bishop |first7=Dorothy V. M. |last8=Nosek |first8=Brian A. |last9=Munafò |first9=Marcus R. |title=A manifesto for reproducible science |journal=Nature Human Behaviour |volume=1 |pages=0021 |language=en |doi=10.1038/s41562-016-0021 |date=January 2017|issue=1 |pmid=33954258 |pmc=7610724 |doi-access=free }}</ref>

=== Redefining significance ===
In 2016, the [[American Statistical Association]] (ASA) published a statement on ''p''-values, saying that "the widespread use of 'statistical significance' (generally interpreted as '''p''&nbsp;≤ 0.05') as a license for making a claim of a scientific finding (or implied truth) leads to considerable distortion of the scientific process".<ref name="Wasserstein 129–133"/> In 2017, a group of 72 authors proposed to enhance reproducibility by changing the ''p''-value threshold for statistical significance from 0.05 to 0.005.<ref>{{Cite journal|last=Benjamin|first=Daniel |display-authors=etal |title=Redefine statistical significance|journal=Nature Human Behaviour|volume=1|issue=1 |pages=6–10|doi=10.1038/s41562-017-0189-z|pmid=30980045 |year=2018 |doi-access=free|hdl=10281/184094|hdl-access=free}}</ref> Other researchers responded that imposing a more stringent significance threshold would aggravate problems such as [[data dredging]]; alternative propositions are thus to select and justify flexible ''p''-value thresholds before collecting data,<ref>{{Cite journal|last=Chawla|first=Dalmeet|date=2017|title='One-size-fits-all' threshold for P values under fire|url=https://www.nature.com/news/one-size-fits-all-threshold-for-p-values-under-fire-1.22625|journal=Nature|doi=10.1038/nature.2017.22625}}</ref> or to interpret ''p''-values as continuous indices, thereby discarding thresholds and statistical significance.<ref>{{cite journal|last1= Amrhein|first1 = Valentin|last2=Greenland |first2=Sander|title=Remove, rather than redefine, statistical significance|journal=Nature Human Behaviour|date=2017|volume=2|issue = 1|page=0224|doi=10.1038/s41562-017-0224-0|pmid = 30980046|s2cid = 46814177}}</ref> Additionally, the change to 0.005 would increase the likelihood of false negatives, whereby the effect being studied is real, but the test fails to show it.<ref>{{cite web |last1=Vyse |first1=Stuart |title=Moving Science's Statistical Goalposts |url=https://www.csicop.org/si/show/moving_sciences_statistical_goal_posts |website=csicop.org |date=November 2017 |publisher=CSI |access-date=10 July 2018}}</ref>

In 2019, over 800 statisticians and scientists signed a message calling for the abandonment of the term "statistical significance" in science,<ref>{{Cite journal|last1=McShane|first1=Blake|last2=Greenland|first2=Sander|last3=Amrhein|first3=Valentin|date=March 2019|title=Scientists rise up against statistical significance|journal=Nature|language=EN|volume=567|issue=7748|pages=305–307|doi=10.1038/d41586-019-00857-9|pmid=30894741|bibcode=2019Natur.567..305A |doi-access=free}}</ref> and the ASA published a further official statement <ref name="Wasserstein2 129–133">{{Cite journal|last1=Wasserstein|first1=Ronald L.|last2=Schirm|first2=Allen L.|last3=Lazar|first3=Nicole A.|date=2019-03-20|title=Moving to a World Beyond "p < 0.05"|journal= [[The American Statistician]]|volume=73|issue=sup1|pages=1–19| doi=10.1080/00031305.2019.1583913|doi-access=free}}</ref> declaring (page 2): {{Blockquote
|text=We conclude, based on our review of the articles in this special issue and the broader literature, that it is time to stop using the term "statistically significant" entirely. Nor should variants such as "significantly different," "<math>p \le 0.05</math>," and "nonsignificant" survive, whether expressed in words, by asterisks in a table, or in some other way.}}