Editing Typical set (section)

==Jointly typical sequences==
Two sequences <math>x^n</math> and <math>y^n</math> are jointly ε-typical if the pair <math>(x^n,y^n)</math> is ε-typical with respect to the joint distribution <math>p(x^n,y^n)=\prod_{i=1}^n p(x_i,y_i)</math> and both <math>x^n</math> and <math>y^n</math> are ε-typical with respect to their marginal distributions <math>p(x^n)</math> and <math>p(y^n)</math>. The set of all such pairs of sequences <math>(x^n,y^n)</math> is denoted by <math>A_{\varepsilon}^n(X,Y)</math>. Jointly ε-typical ''n''-tuple sequences are defined similarly.

Let <math>\tilde{X}^n</math> and <math>\tilde{Y}^n</math> be two independent sequences of random variables with the same marginal distributions <math>p(x^n)</math> and <math>p(y^n)</math>. Then for any ε>0, for sufficiently large ''n'', jointly typical sequences satisfy the following properties:
#<math> P\left[ (X^n,Y^n) \in A_{\varepsilon}^n(X,Y) \right] \geqslant 1 - \epsilon </math>
#<math> \left| A_{\varepsilon}^n(X,Y) \right| \leqslant 2^{n (H(X,Y) + \epsilon)} </math>
#<math> \left| A_{\varepsilon}^n(X,Y) \right| \geqslant (1 - \epsilon) 2^{n (H(X,Y) - \epsilon)} </math>
#<math> P\left[ (\tilde{X}^n,\tilde{Y}^n) \in A_{\varepsilon}^n(X,Y) \right] \leqslant 2^{-n (I(X;Y) - 3 \epsilon)} </math>
#<math> P\left[ (\tilde{X}^n,\tilde{Y}^n) \in A_{\varepsilon}^n(X,Y) \right] \geqslant (1 - \epsilon) 2^{-n (I(X;Y) + 3 \epsilon)}</math>

{{Expand section|date=December 2009}}