Editing Standard deviation (section)

==Interpretation and application==
{{further|Prediction interval|Confidence interval}}
[[File:Comparison standard deviations.svg|thumb|400px|right|Example of samples from two populations with the same mean but different standard deviations. Red population has mean 100 and SD 10; blue population has mean 100 and SD 50.]]
A large standard deviation indicates that the data points can spread far from the mean and a small standard deviation indicates that they are clustered closely around the mean.

For example, each of the three populations {0, 0, 14, 14}, {0, 6, 8, 14} and {6, 6, 8, 8} has a mean of 7. Their standard deviations are 7, 5, and 1, respectively. The third population has a much smaller standard deviation than the other two because its values are all close to 7.  These standard deviations have the same units as the data points themselves. If, for instance, the data set {0, 6, 8, 14} represents the ages of a population of four siblings in years, the standard deviation is 5 years. As another example, the population {1000, 1006, 1008, 1014} may represent the distances traveled by four athletes, measured in meters. It has a mean of 1007 meters, and a standard deviation of 5 meters.

Standard deviation may serve as a measure of uncertainty.  In physical science, for example, the reported standard deviation of a group of repeated [[measurement]]s gives the [[accuracy and precision|precision]] of those measurements.  When deciding whether measurements agree with a theoretical prediction, the standard deviation of those measurements is of crucial importance: if the mean of the measurements is too far away from the prediction (with the distance measured in standard deviations), then the theory being tested probably needs to be revised. This makes sense since they fall outside the range of values that could reasonably be expected to occur if the prediction were correct and the standard deviation appropriately quantified.  See [[prediction interval]].

While the standard deviation does measure how far typical values tend to be from the mean, other measures are available. An example is the [[mean absolute deviation]], which might be considered a more direct measure of average distance, compared to the [[Root-mean-square deviation|root mean square distance]] inherent in the standard deviation.

===Application examples===
The practical value of understanding the standard deviation of a set of values is in appreciating how much variation there is from the average (mean).

====Experiment, industrial and hypothesis testing====
Standard deviation is often used to compare real-world data against a model to test the model.
For example, in industrial applications the weight of products coming off a production line may need to comply with a legally required value. By weighing some fraction of the products an average weight can be found, which will always be slightly different from the long-term average. By using standard deviations, a minimum and maximum value can be calculated that the averaged weight will be within some very high percentage of the time (99.9% or more). If it falls outside the range then the production process may need to be corrected. Statistical tests such as these are particularly important when the testing is relatively expensive. For example, if the product needs to be opened and drained and weighed, or if the product was otherwise used up by the test.

In experimental science, a theoretical model of reality is used. [[Particle physics]] conventionally uses a standard of "'''5 sigma'''" for the declaration of a discovery. A five-sigma level translates to one chance in 3.5 million that a random fluctuation would yield the result. This level of certainty was required in order to assert that a particle consistent with the [[Higgs boson]] had been discovered in two independent experiments at [[CERN]],<ref>{{cite web |url=http://press-archive.web.cern.ch/press-archive/PressReleases/Releases2012/PR17.12E.html |title=CERN experiments observe particle consistent with long-sought Higgs boson &#124; CERN press office |publisher=Press.web.cern.ch |date=4 July 2012 |access-date=30 May 2015 |archive-date=25 March 2016 |archive-url=https://web.archive.org/web/20160325050100/http://press-archive.web.cern.ch/press-archive/PressReleases/Releases2012/PR17.12E.html |url-status=dead }}</ref> also leading to the declaration of the [[first observation of gravitational waves]].<ref>{{Citation|vauthors=((LIGO Scientific Collaboration)), ((Virgo Collaboration))|title=Observation of Gravitational Waves from a Binary Black Hole Merger|journal=Physical Review Letters|volume=116|issue=6|year=2016|pages=061102|doi=10.1103/PhysRevLett.116.061102|arxiv=1602.03837|pmid=26918975|bibcode=2016PhRvL.116f1102A|s2cid=124959784}}</ref>

====Weather====
As a simple example, consider the average daily maximum temperatures for two cities, one inland and one on the coast. It is helpful to understand that the range of daily maximum temperatures for cities near the coast is smaller than for cities inland. Thus, while these two cities may each have the same average maximum temperature, the standard deviation of the daily maximum temperature for the coastal city will be less than that of the inland city as, on any particular day, the actual maximum temperature is more likely to be farther from the average maximum temperature for the inland city than for the coastal one.

====Finance====
In finance, standard deviation is often used as a measure of the [[Risk#Finance|risk]] associated with price-fluctuations of a given asset (stocks, bonds, property, etc.), or the risk of a portfolio of assets<ref>{{cite web|url=http://www.edupristine.com/blog/what-is-standard-deviation |title=What is Standard Deviation |publisher=Pristine |access-date=29 October 2011}}</ref> (actively managed mutual funds, index mutual funds, or ETFs). Risk is an important factor in determining how to efficiently manage a portfolio of investments because it determines the variation in returns on the asset or portfolio and gives investors a mathematical basis for investment decisions (known as [[Modern portfolio theory|mean-variance optimization]]). The fundamental concept of risk is that as it increases, the expected return on an investment should increase as well, an increase known as the risk premium. In other words, investors should expect a higher return on an investment when that investment carries a higher level of risk or uncertainty. When evaluating investments, investors should estimate both the expected return and the uncertainty of future returns. Standard deviation provides a quantified estimate of the uncertainty of future returns.

For example, assume an investor had to choose between two stocks. Stock A over the past 20 years had an average return of 10 percent, with a standard deviation of 20 [[percentage point]]s (pp) and Stock B, over the same period, had average returns of 12 percent but a higher standard deviation of 30 pp. On the basis of risk and return, an investor may decide that Stock A is the safer choice, because Stock B's additional two percentage points of return is not worth the additional 10 pp standard deviation (greater risk or uncertainty of the expected return). Stock B is likely to fall short of the initial investment (but also to exceed the initial investment) more often than Stock A under the same circumstances, and is estimated to return only two percent more on average. In this example, Stock A is expected to earn about 10 percent, plus or minus 20 pp (a range of 30 percent to −10 percent), about two-thirds of the future year returns. When considering more extreme possible returns or outcomes in future, an investor should expect results of as much as 10 percent plus or minus 60 pp, or a range from 70 percent to −50 percent, which includes outcomes for three standard deviations from the average return (about 99.7 percent of probable returns).

Calculating the average (or arithmetic mean) of the return of a security over a given period will generate the expected return of the asset. For each period, subtracting the expected return from the actual return results in the difference from the mean.  Squaring the difference in each period and taking the average gives the overall variance of the return of the asset.  The larger the variance, the greater risk the security carries.  Finding the square root of this variance will give the standard deviation of the investment tool in question.

Financial time series are known to be non-stationary series, whereas the statistical calculations above, such as standard deviation, apply only to stationary series. To apply the above statistical tools to non-stationary series, the series first must be transformed to a stationary series, enabling use of statistical tools that now have a valid basis from which to work.

===Geometric interpretation===
To gain some geometric insights and clarification, we will start with a population of three values, {{math|{{var|x}}{{sub|1}}, {{var|x}}{{sub|2}}, {{var|x}}{{sub|3}}}}. This defines a point {{math|1={{var|P}} = ({{var|x}}{{sub|1}}, {{var|x}}{{sub|2}}, {{var|x}}{{sub|3}})}} in {{math|'''R'''{{sup|3}}}}. Consider the line {{math|1={{var|L}} = {{mset|({{var|r}}, {{var|r}}, {{var|r}}) : {{var|r}} ∈ '''R'''}}}}. This is the "main diagonal" going through the origin. If our three given values were all equal, then the standard deviation would be zero and {{mvar|P}} would lie on {{mvar|L}}. So it is not unreasonable to assume that the standard deviation is related to the ''distance'' of {{mvar|P}} to {{mvar|L}}. That is indeed the case. To move orthogonally from {{mvar|L}} to the point {{mvar|P}}, one begins at the point:

<math display="block">M = \left(\bar{x}, \bar{x}, \bar{x}\right)</math>

whose coordinates are the mean of the values we started out with. 
{{Collapse top|title=Derivation of <math>M = \left(\bar{x}, \bar{x}, \bar{x}\right)</math>}}
<math>M</math> is on <math>L</math> therefore <math>M = (\ell,\ell,\ell)</math> for some <math>\ell \in \mathbb{R}</math>.

The line {{mvar|L}} is to be orthogonal to the vector from {{mvar|M}} to {{mvar|P}}. Therefore:

<math display="block">\begin{align} 
                                       L \cdot (P - M) &= 0    \\[4pt]
  (r, r, r) \cdot (x_1 - \ell, x_2 - \ell, x_3 - \ell) &= 0    \\[4pt]
               r(x_1 - \ell + x_2 - \ell + x_3 - \ell) &= 0    \\[4pt]
                      r\left(\sum_i x_i - 3\ell\right) &= 0    \\[4pt]
                                    \sum_i x_i - 3\ell &= 0    \\[4pt]
                                 \frac{1}{3}\sum_i x_i &= \ell \\[4pt]
                                               \bar{x} &= \ell
\end{align}</math>
{{Collapse bottom}}

A little algebra shows that the distance between {{mvar|P}} and {{mvar|M}} (which is the same as the [[orthogonal distance]] between {{mvar|P}} and the line {{mvar|L}}) <math display="inline">\sqrt{\sum_i \left(x_i - \bar{x}\right)^2}</math> is equal to the standard deviation of the vector {{math|({{var|x}}{{sub|1}}, {{var|x}}{{sub|2}}, {{var|x}}{{sub|3}})}}, multiplied by the square root of the number of dimensions of the vector (3 in this case).

===Chebyshev's inequality===
{{main|Chebyshev's inequality}}
An observation is rarely more than a few standard deviations away from the mean. Chebyshev's inequality ensures that, for all distributions for which the standard deviation is defined, the amount of data within a number of standard deviations of the mean is at least as much as given in the following table.

{|class="wikitable"
|-
! Distance from mean !! Minimum population
|-
| <math>\sqrt{2}\,\sigma</math> || 50%
|-
| <math>2\sigma</math> || 75%
|-
| <math>3\sigma</math> || 89%
|-
| <math>4\sigma</math> || 94%
|-
| <math>5\sigma</math> || 96%
|-
| <math>6\sigma</math> || 97%
|-
| <math>k\sigma</math> || <math>1 - \frac{1}{k^2}</math><ref>{{cite book|last=Ghahramani|first=Saeed|year=2000|title=Fundamentals of Probability|url=https://archive.org/details/fundamentalsprob00ghah_271|url-access=limited|edition=2nd|publisher=Prentice Hall|location=New Jersey|page=[https://archive.org/details/fundamentalsprob00ghah_271/page/n445 438]|isbn=9780130113290 }}</ref>
|-
| <math>\frac{1}{\sqrt{1 - \ell}}\, \sigma</math> || <math>\ell</math>
|}

===Rules for normally distributed data===
[[File:Standard deviation diagram.svg|thumb|Dark blue is one standard deviation on either side of the mean. For the normal distribution, this accounts for 68.27 percent of the set; while two standard deviations from the mean (medium and dark blue) account for 95.45 percent; three standard deviations (light, medium, and dark blue) account for 99.73 percent; and four standard deviations account for 99.994 percent. The two points of the curve that are one standard deviation from the mean are also the [[inflection point]]s.]]

The [[central limit theorem]] states that the distribution of an average of many independent, identically distributed random variables tends toward the famous bell-shaped normal distribution with a [[probability density function]] of

<math display="block">f\left(x, \mu, \sigma^2\right) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{1}{2}\left(\frac{x - \mu}{\sigma}\right)^2}</math>

where {{mvar|μ}} is the [[expected value]] of the random variables, {{mvar|σ}} equals their distribution's standard deviation divided by {{math|{{var|n}}{{sup|{{frac|1|2}}}}}}, and {{mvar|n}} is the number of random variables. The standard deviation therefore is simply a scaling variable that adjusts how broad the curve will be, though it also appears in the [[normalizing constant]].

If a data distribution is approximately normal, then the proportion of data values within {{mvar|z}} standard deviations of the mean is defined by:

<math display="block">\text{Proportion} = \operatorname{erf}\left(\frac{z}{\sqrt{2}}\right)</math>

where <math>\textstyle\operatorname{erf}</math> is the [[error function]]. The proportion that is less than or equal to a number, {{mvar|x}}, is given by the [[cumulative distribution function]]:<ref>{{cite web |url= http://mathworld.wolfram.com/DistributionFunction.html |author= Eric W. Weisstein |title= Distribution Function |work=MathWorld |publisher=Wolfram |access-date= 30 September 2014}}</ref>

<math display="block">\text{Proportion} \le x = \frac{1}{2}\left[1 + \operatorname{erf}\left(\frac{x - \mu}{\sigma\sqrt{2}}\right)\right] = \frac{1}{2}\left[1 + \operatorname{erf}\left(\frac{z}{\sqrt{2}}\right)\right].</math>

If a data distribution is approximately normal then about 68 percent of the data values are within one standard deviation of the mean (mathematically, {{math|{{var|μ}} ± {{var|σ}}}}, where {{mvar|μ}} is the arithmetic mean), about 95 percent are within two standard deviations ({{math|{{var|μ}} ± 2{{var|σ}}}}), and about 99.7 percent lie within three standard deviations ({{math|{{var|μ}} ± 3{{var|σ}}}}). This is known as the ''[[68–95–99.7 rule]]'', or ''the empirical rule''.

For various values of {{mvar|z}}, the percentage of values expected to lie in and outside the symmetric interval, {{math|{{var|CI}} {{=}} (−{{var|z}}{{var|σ}}, {{var|z}}{{var|σ}})}}, are as follows:
[[File:Confidence interval by Standard deviation.svg|thumb|Percentage within(''z'')]]
[[File:Standard deviation by Confidence interval.svg|thumb|''z''(Percentage within)]]
{{anchor|Table}}
{| class="wikitable" style="font-size:&nbsp;"
|-
! rowspan=2 | Confidence <br />interval
! Proportion within
! colspan=2 | Proportion without
|-
! Percentage
! Percentage
! Fraction
|-
| {{val|0.318639}}{{mvar|σ}}
| 25%
| 75%
| 3 / 4
|-
| {{val|0.674490}}{{mvar|σ}}
| {{val|50}}%
| {{val|50}}%
| 1&nbsp;/&nbsp;{{val|2}}
|-
| {{val|{{#expr:0.97792452561403 round 6}}}}{{mvar|σ}}
| 66.6667%
| 33.3333%
| 1&nbsp;/&nbsp;3
|-
| {{val|0.994458}}{{mvar|σ}}
| 68%
| 32%
| 1&nbsp;/&nbsp;3.125
|-
| 1{{mvar|σ}}
| {{val|68.2689492}}%
| {{val|31.7310508}}%
| 1&nbsp;/&nbsp;{{val|3.1514872}}
|-
| {{val|1.281552}}{{mvar|σ}}
| 80%
| 20%
| 1&nbsp;/&nbsp;5
|-
| {{val|1.644854}}{{mvar|σ}}
| 90%
| 10%
| 1&nbsp;/&nbsp;10
|-
| {{val|1.959964}}{{mvar|σ}}
| 95%
| 5%
| 1&nbsp;/&nbsp;20
|-
| 2{{mvar|σ}}
| {{val|95.4499736}}%
| {{val|4.5500264}}%
| 1&nbsp;/&nbsp;{{val|21.977895}}
|-
| {{val|2.575829}}{{mvar|σ}}
| 99%
| 1%
| 1&nbsp;/&nbsp;100
|-
| 3{{mvar|σ}}
| {{val|99.7300204}}%
| {{val|0.2699796}}%
| 1&nbsp;/&nbsp;370.398
|-
| {{val|3.290527}}{{mvar|σ}}
| 99.9%
| 0.1%
| 1&nbsp;/&nbsp;{{val|1000}}
|-
| {{val|3.890592}}{{mvar|σ}}
| 99.99%
| 0.01%
| 1&nbsp;/&nbsp;{{val|10000}}
|-
| 4{{mvar|σ}}
| {{val|99.993666}}%
| {{val|0.006334}}%
| 1&nbsp;/&nbsp;{{val|15787}}
|-
| {{val|4.417173}}{{mvar|σ}}
| 99.999%
| 0.001%
| 1&nbsp;/&nbsp;{{val|100000}}
|-
| {{val|4.5}}{{mvar|σ}}
| {{gaps|99.999|320|465|3751%}}
| {{gaps|0.000|679|534|6249%}}
| 1&nbsp;/&nbsp;{{val|147159.5358}}<br />6.8&nbsp;/&nbsp;{{val|1000000}}
|-
| {{val|4.891638}}{{mvar|σ}}
| {{val|99.9999}}%
| {{val|0.0001}}%
| 1&nbsp;/&nbsp;{{val|1000000}}
|-
| 5{{mvar|σ}}
| {{val|99.9999426697}}%
| {{val|0.0000573303}}%
| 1&nbsp;/&nbsp;{{val|1744278}}
|-
| {{val|5.326724}}{{mvar|σ}}
| {{val|99.99999}}%
| {{val|0.00001}}%
| 1&nbsp;/&nbsp;{{val|10000000}}
|-
| {{val|5.730729}}{{mvar|σ}}
| {{val|99.999999}}%
| {{val|0.000001}}%
| 1&nbsp;/&nbsp;{{val|100000000}}
|-
| [[Six Sigma#Sigma levels|{{val|6}}{{mvar|σ}}]]
| {{val|99.9999998027}}%
| {{val|0.0000001973}}%
| 1&nbsp;/&nbsp;{{val|506797346}}
|-
| {{val|6.109410}}{{mvar|σ}}
| {{val|99.9999999}}%
| {{val|0.0000001}}%
| 1&nbsp;/&nbsp;{{val|1000000000}}
|-
| {{val|6.466951}}{{mvar|σ}}
| {{val|99.99999999}}%
| {{val|0.00000001}}%
| 1&nbsp;/&nbsp;{{val|10000000000}}
|-
| {{val|6.806502}}{{mvar|σ}}
| {{val|99.999999999}}%
| {{val|0.000000001}}%
| 1&nbsp;/&nbsp;{{val|100000000000}}
|-
| 7{{mvar|σ}}
| {{gaps|99.999|999|999|7440%}}
| {{val|0.000000000256}}%
| 1&nbsp;/&nbsp;{{val|390682215445}}
|}