Editing Conditional expectation (section)

=== Conditional expectation with respect to a sub-''σ''-algebra ===
[[File:LokaleMittelwertbildung.svg|thumb|upright=1.5|'''Conditional expectation with respect to a ''σ''-algebra:''' in this example the probability space <math>(\Omega, \mathcal{F}, P)</math> is the [0,1] interval with the [[Lebesgue measure]].  We define the following ''σ''-algebras: <math>\mathcal{A} = \mathcal{F}</math>; <math>\mathcal{B}</math> is the ''σ''-algebra generated by the intervals with end-points 0, {{frac|1|4}}, {{frac|1|2}}, {{frac|3|4}}, 1; and <math>\mathcal{C}</math> is the ''σ''-algebra generated by the intervals with end-points 0, {{frac|1|2}}, 1. Here the conditional expectation is effectively the average over the minimal sets of the ''σ''-algebra.]]

Consider the following:
* <math>(\Omega, \mathcal{F}, P)</math> is a [[probability space]].
* <math>X\colon\Omega \to \mathbb{R}^n</math> is a [[random variable#Definition|random variable]] on that probability space with finite expectation.
* <math>\mathcal{H} \subseteq \mathcal{F}</math> is a sub-[[sigma-algebra|''σ''-algebra]] of <math>\mathcal{F}</math>.

Since <math>\mathcal{H}</math> is a sub <math>\sigma</math>-algebra of <math>\mathcal{F}</math>, the function <math>X\colon\Omega \to \mathbb{R}^n</math> is usually not <math>\mathcal{H}</math>-measurable, thus the existence of the integrals of the form <math display="inline">\int_H X \,dP|_\mathcal{H}</math>, where <math>H\in\mathcal{H}</math> and <math>P|_\mathcal{H}</math> is the restriction of <math>P</math> to <math>\mathcal{H}</math>, cannot be stated in general. However, the local averages <math display="inline">\int_H X\,dP</math> can be recovered in <math>(\Omega, \mathcal{H}, P|_\mathcal{H})</math> with the help of the conditional expectation. 

A '''conditional expectation''' of ''X'' given <math>\mathcal{H}</math>, denoted as <math>\operatorname{E}(X\mid\mathcal{H})</math>, is any <math>\mathcal{H}</math>-[[measurable function]] <math>\Omega \to \mathbb{R}^n</math> which satisfies:

:<math> \int_H\operatorname{E}(X \mid \mathcal{H})\,\mathrm{d}P = \int_H X \,\mathrm{d}P</math>

for each <math>H \in \mathcal{H}</math>.<ref name=billingsley1995/>

As noted in the <math>L^2</math> discussion, this condition is equivalent to saying that the [[residual (statistics)|residual]] <math>X - \operatorname{E}(X \mid \mathcal{H})</math> is orthogonal to the indicator functions <math>1_H</math>:
:<math> \langle X - \operatorname{E}(X \mid \mathcal{H}), 1_H \rangle = 0 </math>

==== Existence ====

The existence of <math>\operatorname{E}(X\mid\mathcal{H})</math> can be established by noting that <math display="inline">\mu^X\colon F \mapsto \int_F X \, \mathrm{d}P</math> for <math>F \in \mathcal{F}</math> is a finite measure on <math>(\Omega, \mathcal{F})</math> that is [[absolute continuity|absolutely continuous]] with respect to  <math>P</math>.  If <math>h</math> is the [[natural injection]] from <math>\mathcal{H}</math> to <math>\mathcal{F}</math>, then <math>\mu^X \circ h = \mu^X|_\mathcal{H}</math> is the restriction of <math>\mu^X</math> to <math>\mathcal{H}</math> and <math>P \circ h = P|_\mathcal{H}</math> is the restriction of <math>P</math> to <math>\mathcal{H}</math>.  Furthermore, <math>\mu^X \circ h</math> is absolutely continuous with respect to <math>P \circ h</math>, because the condition
:<math>P \circ h (H) = 0 \iff P(h(H)) = 0</math>
implies
:<math>\mu^X(h(H)) = 0 \iff \mu^X \circ h(H) = 0.</math>

Thus, we have 
:<math>\operatorname{E}(X\mid\mathcal{H}) = \frac{\mathrm{d}\mu^X|_\mathcal{H}}{\mathrm{d}P|_\mathcal{H}} = \frac{\mathrm{d}(\mu^X \circ h)}{\mathrm{d}(P \circ h)},</math>
where the derivatives are [[Radon–Nikodym theorem|Radon–Nikodym derivatives]] of measures.

==== Conditional expectation with respect to a random variable ====
Consider, in addition to the above,
* A [[measurable space]] <math>(U, \Sigma)</math>, and
* A random variable <math>Y\colon\Omega \to U</math>.

The conditional expectation of {{mvar|X}} given {{mvar|Y}} is defined by applying the above construction on the [[Σ-algebra#σ-algebra generated by random variable or vector|''σ''-algebra generated by]] {{mvar|Y}}:
:<math>\operatorname{E}[X\mid Y] := \operatorname{E}[X\mid\sigma(Y)]. </math>

By the [[Doob–Dynkin lemma]], there exists a function <math>e_X \colon U \to \mathbb{R}^n</math> such that
:<math>\operatorname{E}[X\mid Y] = e_X(Y). </math>

==== Discussion ====

* This is not a constructive definition; we are merely given the required property that a conditional expectation must satisfy.
** The definition of <math>\operatorname{E}(X \mid \mathcal{H})</math> may resemble that of <math>\operatorname{E}(X \mid H)</math> for an event <math>H</math> but these are very different objects.  The former is a <math>\mathcal{H}</math>-measurable function <math>\Omega \to \mathbb{R}^n</math>, while the latter is an element of <math>\mathbb{R}^n</math> and <math>\operatorname{E}(X \mid H)\ P(H)= \int_H X \,\mathrm{d}P= \int_H \operatorname{E} (X\mid\mathcal{H})\,\mathrm{d}P</math> for <math>H\in\mathcal{H}</math>.
** Uniqueness can be shown to be [[almost surely|almost sure]]: that is, versions of the same conditional expectation will only differ on a [[null set|set of probability zero]].
*** Often, one would like to think of <math>\operatorname{E}(X \mid \mathcal{H})</math> as a measure on <math>\Omega</math> for fixed H. For example, it is extremely useful to claim that <math>\sum_i\operatorname{E}(X_i \mid \mathcal{H})</math> is additive for almost all H. However, this does not immediately follow because each <math>\operatorname{E}(X_i \mid \mathcal{H})</math> may have a different null set. Because countable unions of null sets are null sets, for a countable set of <math>X_i</math>, one can choose "versions" of each <math>\operatorname{E}(X_i \mid \mathcal{H})</math> with aligned null sets as to maintain additivity for almost all H. However, to align the "null sets of dysfunction" of <math>\operatorname{E}(X_i \mid \mathcal{H})</math> over all possible <math>X_i</math>, and thus treat <math>\operatorname{E}(X \mid \mathcal{H} = H)</math> as an almost surely unique measure over <math>\Omega</math> (a "regular probability measure"), we need further regularity conditions. Intuitively, to do this, we need to be able to approximate all possible <math>X_i</math> with a countable set of them. This directly corresponds to the conditions for creating a regular probability measure, which are separability and completeness.
* The ''σ''-algebra <math>\mathcal{H}</math> controls the "granularity" of the conditioning.  A conditional expectation <math>E(X\mid\mathcal{H})</math> over a finer (larger) ''σ''-algebra <math>\mathcal{H}</math> retains information about the probabilities of a larger class of events.  A conditional expectation over a coarser (smaller) ''σ''-algebra averages over more events.

==== Conditional probability ====

{{Main|Regular conditional probability}}

For a Borel subset {{mvar|B}} in <math>\mathcal{B}(\mathbb{R}^n)</math>, one can consider the collection of random variables
:<math> \kappa_\mathcal{H}(\omega, B) := \operatorname{E}(1_{X \in B}|\mathcal{H})(\omega). </math>
It can be shown that they form a [[Markov kernel]], that is, for almost all <math>\omega</math>,
<math>\kappa_\mathcal{H}(\omega, -)</math> is a probability measure.<ref>{{cite book |last1=Klenke |first1=Achim |title=Probability theory : a comprehensive course |date=30 August 2013 |location=London |isbn=978-1-4471-5361-0 |edition=Second}}</ref>

The [[Law of the unconscious statistician]] is then
:<math> \operatorname{E}[f(X)\mid\mathcal{H}] = \int f(x) \kappa_\mathcal{H}(-, \mathrm{d}x), </math>
This shows that conditional expectations are, like their unconditional counterparts, integrations,
against a conditional measure.