Editing Variance (section)

==Propagation==

===Addition and multiplication by a constant===
Variance is [[Invariant (mathematics)|invariant]] with respect to changes in a [[location parameter]].  That is, if a constant is added to all values of the variable, the variance is unchanged:
<math display="block">\operatorname{Var}(X+a)=\operatorname{Var}(X).</math>

If all values are scaled by a constant, the variance is [[Homogeneous function|scaled]] by the square of that constant:
<math display="block">\operatorname{Var}(aX)=a^2\operatorname{Var}(X).</math>

The variance of a sum of two random variables is given by
<math display="block">\begin{align}
\operatorname{Var}(aX + bY) &= a^2\operatorname{Var}(X) + b^2\operatorname{Var}(Y) + 2ab\, \operatorname{Cov}(X,Y) \\[1ex]
\operatorname{Var}(aX - bY) &= a^2\operatorname{Var}(X) + b^2\operatorname{Var}(Y) - 2ab\, \operatorname{Cov}(X,Y)
\end{align}</math>

where <math>\operatorname{Cov}(X,Y)</math> is the [[covariance]].

=== Linear combinations ===
In general, for the sum of <math>N</math> random variables <math>\{X_1,\dots,X_N\}</math>, the variance becomes:
<math display="block">\operatorname{Var}\left(\sum_{i=1}^N X_i\right) = \sum_{i,j=1}^N\operatorname{Cov}(X_i,X_j) = \sum_{i=1}^N\operatorname{Var}(X_i) + \sum_{i,j=1,i\ne j}^N\operatorname{Cov}(X_i,X_j),</math> 
see also general [[Bienaymé's identity]].

These results lead to the variance of a [[linear combination]] as:

<math display="block">\begin{align}
\operatorname{Var}\left( \sum_{i=1}^N a_iX_i\right) &=\sum_{i,j=1}^{N} a_ia_j\operatorname{Cov}(X_i,X_j) \\
&= \sum_{i=1}^N a_i^2 \operatorname{Var}(X_i) +   \sum_{i \neq j}            a_i a_j \operatorname{Cov}(X_i,X_j)\\
&= \sum_{i=1}^N a_i^2 \operatorname{Var}(X_i) + 2 \sum_{1 \leq i < j \leq N} a_i a_j \operatorname{Cov}(X_i,X_j).
\end{align}</math>

If the random variables <math>X_1,\dots,X_N</math> are such that
<math display="block">\operatorname{Cov}(X_i,X_j)=0\ ,\ \forall\ (i\ne j) ,</math>
then they are said to be [[Covariance#Definition|uncorrelated]]. It follows immediately from the expression given earlier that if the random variables <math>X_1,\dots,X_N</math> are uncorrelated, then the variance of their sum is equal to the sum of their variances, or, expressed symbolically:

<math display="block">\operatorname{Var}\left(\sum_{i=1}^N X_i\right) = \sum_{i=1}^N\operatorname{Var}(X_i).</math>

Since independent random variables are always uncorrelated (see {{Section link|Covariance|Uncorrelatedness and independence}}), the equation above holds in particular when the random variables <math>X_1,\dots,X_n</math> are independent.  Thus, independence is sufficient but not necessary for the variance of the sum to equal the sum of the variances.

====Matrix notation for the variance of a linear combination====

Define <math>X</math> as a column vector of <math>n</math> random variables <math>X_1, \ldots,X_n</math>, and <math>c</math> as a column vector of <math>n</math> scalars <math>c_1, \ldots,c_n</math>. Therefore, <math>c^\mathsf{T} X</math> is a [[linear combination]] of these random variables, where <math>c^\mathsf{T}</math> denotes the [[transpose]] of <math>c</math>. Also let <math>\Sigma</math> be the [[covariance matrix]] of <math>X</math>. The variance of <math>c^\mathsf{T}X</math> is then given by:<ref>{{Cite book | last1=Johnson | first1=Richard | last2=Wichern | first2=Dean | year=2001 | title=Applied Multivariate Statistical Analysis | url=https://archive.org/details/appliedmultivari00john_130 | url-access=limited | publisher=Prentice Hall | page=[https://archive.org/details/appliedmultivari00john_130/page/n96 76] | isbn=0-13-187715-1 }}</ref>

<math display="block">\operatorname{Var}\left(c^\mathsf{T} X\right) = c^\mathsf{T} \Sigma c .</math>

This implies that the variance of the mean can be written as (with a column vector of ones)

<math display="block">\operatorname{Var}\left(\bar{x}\right) = \operatorname{Var}\left(\frac{1}{n} 1'X\right) = \frac{1}{n^2} 1'\Sigma 1.</math>

===Sum of variables===

====Sum of uncorrelated variables====
{{main article|Bienaymé's identity}}
{{see also|Sum of normally distributed random variables}}
One reason for the use of the variance in preference to other measures of dispersion is that the variance of the sum (or the difference) of [[uncorrelated]] random variables is the sum of their variances:

<math display="block">\operatorname{Var}\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n \operatorname{Var}(X_i).</math>

This statement is called the [[Irénée-Jules Bienaymé|Bienaymé]] formula<ref>[[Michel Loève|Loève, M.]] (1977) "Probability Theory", ''Graduate Texts in Mathematics'', Volume 45, 4th edition, Springer-Verlag, p.&nbsp;12.</ref> and was discovered in 1853.<ref>[[Irénée-Jules Bienaymé|Bienaymé, I.-J.]] (1853) "Considérations à l'appui de la découverte de Laplace sur la loi de probabilité dans la méthode des moindres carrés", ''Comptes rendus de l'Académie des sciences Paris'', 37, p.&nbsp;309–317; digital copy available [http://visualiseur.bnf.fr/CadresFenetre?O=NUMM-2994&I=313] {{Webarchive|url=https://web.archive.org/web/20180623145935/http://visualiseur.bnf.fr/CadresFenetre?O=NUMM-2994&I=313|date=2018-06-23}}</ref><ref>[[Irénée-Jules Bienaymé|Bienaymé, I.-J.]] (1867) "Considérations à l'appui de la découverte de Laplace sur la loi de probabilité dans la méthode des moindres carrés", ''Journal de Mathématiques Pures et Appliquées, Série 2'', Tome 12, p.&nbsp;158–167; digital copy available [http://gallica.bnf.fr/ark:/12148/bpt6k16411c/f166.image.n19][http://sites.mathdoc.fr/JMPA/PDF/JMPA_1867_2_12_A10_0.pdf]</ref> It is often made with the stronger condition that the variables are [[statistical independence|independent]], but being uncorrelated suffices. So if all the variables have the same variance σ<sup>2</sup>, then, since division by ''n'' is a linear transformation, this formula immediately implies that the variance of their mean is

<math display="block">
  \operatorname{Var}\left(\overline{X}\right) = \operatorname{Var}\left(\frac{1}{n} \sum_{i=1}^n X_i\right) =
  \frac{1}{n^2}\sum_{i=1}^n \operatorname{Var}\left(X_i\right) =
  \frac{1}{n^2}n\sigma^2 =
  \frac{\sigma^2}{n}.
</math>

That is, the variance of the mean decreases when ''n'' increases. This formula for the variance of the mean is used in the definition of the [[standard error (statistics)|standard error]] of the sample mean, which is used in the [[central limit theorem]].

To prove the initial statement, it suffices to show that

<math display="block">\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y).</math>

The general result then follows by induction.  Starting with the definition,

<math display="block">\begin{align}
  \operatorname{Var}(X + Y) &= \operatorname{E}\left[(X + Y)^2\right] - (\operatorname{E}[X + Y])^2 \\[5pt]
    &= \operatorname{E}\left[X^2 + 2XY + Y^2\right] - (\operatorname{E}[X] + \operatorname{E}[Y])^2.
\end{align}</math>

Using the linearity of the [[Expectation Operator|expectation operator]] and the assumption of independence (or uncorrelatedness) of ''X'' and ''Y'', this further simplifies as follows:

<math display="block">\begin{align}
   \operatorname{Var}(X + Y) &= \operatorname{E}{\left[X^2\right]} + 2\operatorname{E}[XY] + \operatorname{E}{\left[Y^2\right]} - \left(\operatorname{E}[X]^2 + 2\operatorname{E}[X] \operatorname{E}[Y] + \operatorname{E}[Y]^2\right) \\[5pt]
   &= \operatorname{E}\left[X^2\right] + \operatorname{E}\left[Y^2\right] - \operatorname{E}[X]^2 - \operatorname{E}[Y]^2 \\[5pt]
   &= \operatorname{Var}(X) + \operatorname{Var}(Y).
\end{align}</math>

====Sum of correlated variables====

=====Sum of correlated variables with fixed sample size=====
{{main article|Bienaymé's identity}}
In general, the variance of the sum of {{math|n}} variables is the sum of their [[covariance]]s:

<math display="block">\operatorname{Var}\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n \sum_{j=1}^n \operatorname{Cov}\left(X_i, X_j\right) = \sum_{i=1}^n \operatorname{Var}\left(X_i\right) + 2 \sum_{1 \leq i < j\leq n} \operatorname{Cov}\left(X_i, X_j\right).</math>

(Note: The second equality comes from the fact that {{math|1=Cov(''X''<sub>''i''</sub>,''X''<sub>''i''</sub>) = Var(''X''<sub>''i''</sub>)}}.)

Here, <math>\operatorname{Cov}(\cdot,\cdot)</math> is the [[covariance]], which is zero for independent random variables (if it exists). The formula states that the variance of a sum is equal to the sum of all elements in the covariance matrix of the components. The next expression states equivalently that the variance of the sum is the sum of the diagonal of covariance matrix plus two times the sum of its upper triangular elements (or its lower triangular elements); this emphasizes that the covariance matrix is symmetric. This formula is used in the theory of [[Cronbach's alpha]] in [[classical test theory]].

So, if the variables have equal variance ''σ''<sup>2</sup> and the average [[correlation]] of distinct variables is ''ρ'', then the variance of their mean is

<math display="block">\operatorname{Var}\left(\overline{X}\right) = \frac{\sigma^2}{n} + \frac{n - 1}{n}\rho\sigma^2.</math>

This implies that the variance of the mean increases with the average of the correlations. In other words, additional correlated observations are not as effective as additional independent observations at reducing the [[standard error|uncertainty of the mean]]. Moreover, if the variables have unit variance, for example if they are standardized, then this simplifies to

<math display="block">\operatorname{Var}\left(\overline{X}\right) = \frac{1}{n} + \frac{n - 1}{n}\rho.</math>

This formula is used in the [[Spearman–Brown prediction formula]] of classical test theory. This converges to ''ρ'' if ''n'' goes to infinity, provided that the average correlation remains constant or converges too. So for the variance of the mean of standardized variables with equal correlations or converging average correlation we have

<math display="block">\lim_{n \to \infty} \operatorname{Var}\left(\overline{X}\right) = \rho.</math>

Therefore, the variance of the mean of a large number of standardized variables is approximately equal to their average correlation. This makes clear that the sample mean of correlated variables does not generally converge to the population mean, even though the [[law of large numbers]] states that the sample mean will converge for independent variables.

=====Sum of uncorrelated variables with random sample size=====
There are cases when a sample is taken without knowing, in advance, how many observations will be acceptable according to some criterion. In such cases, the sample size {{math|N}} is a random variable whose variation adds to the variation of {{math|X}}, such that,<ref>Cornell, J R, and Benjamin, C A, ''Probability, Statistics, and Decisions for Civil Engineers,'' McGraw-Hill, NY, 1970, pp.178-9.</ref>
<math display="block">\operatorname{Var}\left(\sum_{i=1}^{N}X_i\right)=\operatorname{E}\left[N\right]\operatorname{Var}(X)+\operatorname{Var}(N)(\operatorname{E}\left[X\right])^2</math>
which follows from the [[law of total variance]].

If {{math|N}} has a [[Poisson distribution]], then <math>\operatorname{E}[N]=\operatorname{Var}(N)</math> with estimator {{math|n}} = {{math|N}}. So, the estimator of <math>\operatorname{Var}\left(\sum_{i=1}^{n}X_i\right)</math> becomes <math>n{S_x}^2+n\bar{X}^2</math>, giving <math>\operatorname{SE}(\bar{X})=\sqrt{\frac{{S_x}^2+\bar{X}^2}{n}}</math>
(see [[Standard error#Standard_error_of_the_sample_mean|standard error of the sample mean]]).

====Weighted sum of variables====
{{see also|Weighted arithmetic mean#Variance{{!}}Variance of a weighted arithmetic mean}}
{{distinguish|Weighted variance}}

The scaling property and the Bienaymé formula, along with the property of the [[covariance]] {{math|Cov(''aX'',&nbsp;''bY'') {{=}} ''ab'' Cov(''X'',&nbsp;''Y'')}}  jointly imply that

<math display="block">\operatorname{Var}(aX \pm bY) =a^2 \operatorname{Var}(X) + b^2 \operatorname{Var}(Y) \pm 2ab\, \operatorname{Cov}(X, Y).</math>

This implies that in a weighted sum of variables, the variable with the largest weight will have a disproportionally large weight in the variance of the total. For example, if ''X'' and ''Y'' are uncorrelated and the weight of ''X'' is two times the weight of ''Y'', then the weight of the variance of ''X'' will be four times the weight of the variance of ''Y''.

The expression above can be extended to a weighted sum of multiple variables:

<math display="block">\operatorname{Var}\left(\sum_{i}^n a_iX_i\right) = \sum_{i=1}^na_i^2 \operatorname{Var}(X_i) + 2\sum_{1\le i}\sum_{<j\le n}a_ia_j\operatorname{Cov}(X_i,X_j)</math>

===Product of variables===

====Product of independent variables====

If two variables X and Y are [[Independence (probability theory)|independent]], the variance of their product is given by<ref>{{Cite journal |last=Goodman |first=Leo A. |author-link=Leo Goodman |date=December 1960 |title=On the Exact Variance of Products |journal=Journal of the American Statistical Association |volume=55 |issue=292 |pages=708–713 |doi=10.2307/2281592 |jstor=2281592}}</ref>
<math display="block">\operatorname{Var}(XY) = [\operatorname{E}(X)]^2 \operatorname{Var}(Y) + [\operatorname{E}(Y)]^2 \operatorname{Var}(X) + \operatorname{Var}(X)\operatorname{Var}(Y).</math>

Equivalently, using the basic properties of expectation, it is given by

<math display="block">\operatorname{Var}(XY) = \operatorname{E}\left(X^2\right) \operatorname{E}\left(Y^2\right) - [\operatorname{E}(X)]^2 [\operatorname{E}(Y)]^2.</math>

====Product of statistically dependent variables====

In general, if two variables are statistically dependent, then the variance of their product is given by:
<math display="block">\begin{align}
  \operatorname{Var}(XY)
    ={} &\operatorname{E}\left[X^2 Y^2\right] - [\operatorname{E}(XY)]^2 \\[5pt]
    ={} &\operatorname{Cov}\left(X^2, Y^2\right) + \operatorname{E}(X^2)\operatorname{E}\left(Y^2\right) - [\operatorname{E}(XY)]^2 \\[5pt]
    ={} &\operatorname{Cov}\left(X^2, Y^2\right) + \left(\operatorname{Var}(X) + [\operatorname{E}(X)]^2\right) \left(\operatorname{Var}(Y) + [\operatorname{E}(Y)]^2\right) \\[5pt]
        &- [\operatorname{Cov}(X, Y) + \operatorname{E}(X)\operatorname{E}(Y)]^2
\end{align}</math>

===Arbitrary functions===
{{main|Uncertainty propagation}}

The [[delta method]] uses second-order [[Taylor expansion]]s to approximate the variance of a function of one or more random variables: see [[Taylor expansions for the moments of functions of random variables]]. For example, the approximate variance of a function of one variable is given by

<math display="block">\operatorname{Var}\left[f(X)\right] \approx \left(f'(\operatorname{E}\left[X\right])\right)^2\operatorname{Var}\left[X\right]</math>

provided that ''f'' is twice differentiable and that the mean and variance of ''X'' are finite.