Editing Typical set (section)

===Example===
Counter-intuitively, the most likely sequence is often not a member of the typical set. For example, suppose that ''X'' is an i.i.d [[Bernoulli_distribution|Bernoulli random variable]] with ''p''(0)=0.1 and ''p''(1)=0.9. In ''n'' independent trials, since ''p''(1)>''p''(0), the most likely sequence of outcome is the sequence of all 1's, (1,1,...,1). Here the entropy of ''X'' is ''H''(''X'')=0.469, while 

:<math> -\frac{1}{n}\log_2 p\left(x^{(n)}=(1,1,\ldots,1)\right) = -\frac{1}{n}\log_2 (0.9^n) = 0.152</math>

So this sequence is not in the typical set because its average logarithmic probability cannot come arbitrarily close to the entropy of the random variable ''X'' no matter how large we take the value of ''n''. 

For Bernoulli random variables, the typical set consists of sequences with average numbers of 0s and 1s in ''n'' independent trials. This is easily demonstrated: If ''p(1) = p'' and ''p(0) = 1-p'', then for ''n'' trials with ''m'' 1's, we have 

:<math> -\frac{1}{n} \log_2 p(x^{(n)}) = -\frac{1}{n} \log_2 p^m (1-p)^{n-m} = -\frac{m}{n} \log_2 p - \left( \frac{n-m}{n} \right) \log_2 (1-p).</math>

The average number of 1's in a sequence of Bernoulli trials is ''m = np''. Thus, we have 

:<math> -\frac{1}{n} \log_2 p(x^{(n)}) = - p \log_2 p - (1-p) \log_2 (1-p) = H(X).</math>

For this example, if ''n''=10, then the typical set consist of all sequences that have a single 0 in the entire sequence. In case ''p''(0)=''p''(1)=0.5, then every possible binary sequences belong to the typical set.