Editing Order statistic (section)

== Application: confidence intervals for quantiles ==

An interesting question is how well the order statistics perform as estimators of the [[quantile]]s of the underlying distribution.

=== A small-sample-size example ===

The simplest case to consider is how well the sample median estimates the population median.

As an example, consider a random sample of size 6. In that case, the sample median is usually defined as the midpoint of the interval delimited by the 3rd and 4th order statistics. However, we know from the preceding discussion that the probability that this interval actually contains the population median is {{Clarify|reason=In which way does the "preceding discussion" explain that the probability is that one?|date=September 2021}}

:<math>{6\choose 3}(1/2)^{6} = {5\over 16} \approx 31\%.</math>

Although the sample median is probably among the best distribution-independent [[point estimate]]s of the population median, what this example illustrates is that it is not a particularly good one in absolute terms. In this particular case, a better confidence interval for the median is the one delimited by the 2nd and 5th order statistics, which contains the population median with probability

:<math>\left[{6\choose 2}+{6\choose 3}+{6\choose 4}\right](1/2)^{6} = {25\over 32} \approx 78\%.</math>

With such a small sample size, if one wants at least 95% confidence, one is reduced to saying that the median is between the minimum and the maximum of the 6 observations with probability 31/32 or approximately 97%. Size 6 is, in fact, the smallest sample size such that the interval determined by the minimum and the maximum is at least a 95% confidence interval for the population median.
=== Large sample sizes ===

For the uniform distribution, as ''n'' tends to infinity, the ''p''<sup>th</sup> sample quantile is asymptotically [[normal distribution|normally distributed]], since it is approximated by

: <math>U_{(\lceil np \rceil)} \sim AN\left(p,\frac{p(1-p)}{n}\right).</math>

For a general distribution ''F'' with a continuous non-zero density at ''F''<sup>&nbsp;−1</sup>(''p''), a similar asymptotic normality applies:

: <math>X_{(\lceil np \rceil)} \sim AN\left(F^{-1}(p),\frac{p(1-p)}{n[f(F^{-1}(p))]^2}\right)</math>

where ''f'' is the [[density function]], and ''F''<sup>&nbsp;−1</sup> is the [[quantile function]] associated with ''F''. One of the first people to mention and prove this result was [[Frederick Mosteller]] in his seminal paper in 1946.<ref name = "Mosteller">{{cite journal|last = Mosteller| first = Frederick| author-link = Frederick Mosteller| year = 1946| title = On Some Useful "Inefficient" Statistics| url = http://projecteuclid.org/euclid.aoms/1177730881| journal = [[Annals of Mathematical Statistics]]| volume = 17| issue = 4| pages = 377–408| doi = 10.1214/aoms/1177730881| access-date = February 26, 2015| doi-access = free}}</ref> Further research led in the 1960s to the [[Raghu Raj Bahadur|Bahadur]] representation which provides information about the errorbounds. The convergence to normal distribution also holds in a stronger sense, such as convergence in [[Kullback–Leibler divergence|relative entropy or KL divergence]].<ref>M. Cardone, A. Dytso and C. Rush, "Entropic Central Limit Theorem for Order Statistics," in IEEE Transactions on Information Theory, vol. 69, no. 4, pp. 2193-2205, April 2023, doi: 10.1109/TIT.2022.3219344.</ref>

An interesting observation can be made in the case where the distribution is symmetric, and the population median equals the population mean. In this case, the [[sample mean]], by the [[central limit theorem]], is also asymptotically normally distributed, but with variance σ<sup>2</sup>''/n'' instead. This asymptotic analysis suggests that the mean outperforms the median in cases of low [[kurtosis]], and vice versa. For example, the median achieves better confidence intervals for the [[Laplace distribution]], while the mean performs better for ''X'' that are normally distributed.

==== Proof ====
It can be shown that

: <math>B(k,n+1-k)\ \stackrel{\mathrm{d}}{=}\ \frac{X}{X + Y},</math>

where

: <math> X = \sum_{i=1}^{k} Z_i, \quad Y = \sum_{i=k+1}^{n+1} Z_i,</math>

with ''Z<sub>i</sub>'' being independent identically distributed [[exponential distribution|exponential]] random variables with rate 1. Since ''X''/''n'' and ''Y''/''n'' are asymptotically normally distributed by the CLT, our results follow by application of the [[delta method]].