Editing P-value (section)

== Basic concepts ==
In statistics, every conjecture concerning the unknown [[probability distribution]] of a collection of random variables representing the observed data <math>X</math> in some study is called a ''statistical hypothesis''. If we state one hypothesis only and the aim of the statistical test is to see whether this hypothesis is tenable, but not to investigate other specific hypotheses, then such a test is called a [[Statistical hypothesis testing|null hypothesis test]].

As our statistical hypothesis will, by definition, state some property of the distribution, the [[null hypothesis]] is the default hypothesis under which that property does not exist. The null hypothesis is typically that some parameter (such as a correlation or a difference between means) in the populations of interest is zero. Our hypothesis might specify the probability distribution of <math>X</math> precisely, or it might only specify that it belongs to some class of distributions. Often, we reduce the data to a single numerical statistic, e.g., <math>T</math>, whose marginal probability distribution is closely connected to a main question of interest in the study.

The ''p''-value is used in the context of null hypothesis testing in order to quantify the [[statistical significance]] of a result, the result being the observed value of the chosen statistic <math>T</math>.{{NoteTag|The statistical significance of a result does not imply that the result also has real-world relevance. For instance, a medication might have a statistically significant effect that is too small to be interesting.}} The lower the ''p''-value is, the lower the probability of getting that result if the null hypothesis were true. A result is said to be ''statistically significant'' if it allows us to reject the null hypothesis. All other things being equal, smaller ''p''-values are taken as stronger evidence against the null hypothesis.

Loosely speaking, rejection of the null hypothesis implies that there is sufficient evidence against it.

As a particular example, if a null hypothesis states that a certain summary statistic <math>T</math> follows the standard [[normal distribution]] <math>\mathcal N(0, 1),</math> then the rejection of this null hypothesis could mean that (i) the mean of <math>T</math> is not 0, or (ii) the [[variance]] of <math>T</math> is not 1, or (iii) <math>T</math> is not normally distributed. Different tests of the same null hypothesis would be more or less sensitive to different alternatives. However, even if we do manage to reject the null hypothesis for all 3 alternatives, and even if we know that the distribution is normal and variance is 1, the null hypothesis test does not tell us which non-zero values of the mean are now most plausible. The more independent observations from the same probability distribution one has, the more accurate the test will be, and the higher the precision with which one will be able to determine the mean value and show that it is not equal to zero; but this will also increase the importance of evaluating the real-world or scientific relevance of this deviation.