Editing P-value (section)

==== Optional stopping ====
{{Anchor|Optional stopping}}

The difference between the two meanings of "extreme" appear when we consider a sequential hypothesis testing, or optional stopping, for the fairness of the coin. In general, optional stopping changes how p-value is calculated.<ref>{{Cite journal |last=Goodman |first=Steven |date=2008-07-01 |title=A Dirty Dozen: Twelve P-Value Misconceptions |url=https://www.sciencedirect.com/science/article/pii/S0037196308000620 |journal=Seminars in Hematology |series=Interpretation of Quantitative Research |volume=45 |issue=3 |pages=135–140 |doi=10.1053/j.seminhematol.2008.04.003 |pmid=18582619 |issn=0037-1963}}</ref><ref>{{Cite journal |last=Wagenmakers |first=Eric-Jan |date=October 2007 |title=A practical solution to the pervasive problems of p values |url=http://link.springer.com/10.3758/BF03194105 |journal=Psychonomic Bulletin & Review |language=en |volume=14 |issue=5 |pages=779–804 |doi=10.3758/BF03194105 |pmid=18087943 |issn=1069-9384}}</ref> Suppose we design the experiment as follows:
* Flip the coin twice. If both comes up heads or tails, end the experiment.
* Else, flip the coin 4 more times.

This experiment has 7 types of outcomes: 2&nbsp;heads, 2&nbsp;tails, 5&nbsp;heads 1&nbsp;tail,&nbsp;..., 1&nbsp;head 5&nbsp;tails. We now calculate the ''p''-value of the "3&nbsp;heads 3&nbsp;tails" outcome.

If we use the test statistic <math>\text{heads}/\text{tails}</math>, then under the null hypothesis is exactly 1 for two-sided ''p''-value, and exactly <math>19/32</math> for one-sided left-tail ''p''-value, and same for one-sided right-tail ''p''-value.

If we consider every outcome that has equal or lower probability than "3&nbsp;heads 3&nbsp;tails" as "at least as extreme", then the ''p''-value is exactly <math>1/2.</math>

However, suppose we have planned to simply flip the coin 6&nbsp;times no matter what happens, then the second definition of ''p''-value would mean that the ''p''-value of "3&nbsp;heads 3&nbsp;tails" is exactly 1.

Thus, the "at least as extreme" definition of ''p''-value is deeply contextual and depends on what the experimenter ''planned'' to do even in situations that did not occur.