Editing Additive white Gaussian noise (section)

=== Coding theorem converse ===
Here we show that rates above the capacity <math>C = \frac {1}{2} \log\left( 1+\frac P N \right)</math> are not achievable.

Suppose that the power constraint is satisfied for a codebook, and further suppose that the messages follow a uniform distribution. Let <math>W</math> be the input messages and <math>\hat{W}</math> the output messages. Thus the information flows as:

<math>W \longrightarrow X^{(n)}(W) \longrightarrow Y^{(n)} \longrightarrow \hat{W}</math>

Making use of [[Fano's inequality]] gives:

<math>H(W\mid\hat{W}) \leq 1+nRP^{(n)}_e = n \varepsilon_n</math> where <math>\varepsilon_n \rightarrow 0</math> as <math>P^{(n)}_e \rightarrow 0</math>

Let <math>X_i</math> be the encoded message of codeword index ''i''. Then:

: <math>
\begin{align}
nR & = H(W) \\
& =I(W;\hat{W}) + H(W\mid\hat{W}) \\
& \leq I(W;\hat{W}) + n\varepsilon_n \\
& \leq I(X^{(n)}; Y^{(n)}) + n\varepsilon_n \\
& = h(Y^{(n)}) - h(Y^{(n)}\mid X^{(n)}) + n\varepsilon_n \\
& = h(Y^{(n)}) - h(Z^{(n)}) + n\varepsilon_n \\
& \leq \sum_{i=1}^n h(Y_i)- h(Z^{(n)}) + n\varepsilon_n \\
& \leq \sum_{i=1}^n I(X_i; Y_i) + n\varepsilon_n
\end{align}
</math>

Let <math>P_i</math> be the average power of the codeword of index i:

:<math>
P_i = \frac{1}{2^{nR}}\sum_{w}x^2_i(w)
\,\!</math>

where the sum is over all input messages <math>w</math>. <math>X_i</math> and <math>Z_i</math> are independent, thus the expectation of the power of <math>Y_i</math> is, for noise level <math>N</math>:

:<math>
E(Y_i^2) = P_i+N
\,\!</math>

And, if <math>Y_i</math> is normally distributed, we have that
:<math>
h(Y_i) \leq \frac{1}{2}\log{2 \pi e} (P_i +N)
\,\!</math>

Therefore,

: <math>
\begin{align}
nR & \leq \sum(h(Y_i)-h(Z_i)) + n \varepsilon_n \\
& \leq \sum \left( \frac{1}{2} \log(2 \pi e (P_i + N)) - \frac{1}{2} \log(2 \pi e N)\right) + n \varepsilon_n \\
& = \sum \frac{1}{2} \log \left(1 + \frac{P_i}N \right) + n \varepsilon_n
\end{align}
</math>

We may apply Jensen's equality to <math>\log(1+x)</math>, a concave (downward) function of ''x'', to get:
:<math>
\frac{1}{n} \sum_{i=1}^n \frac{1}{2}\log\left(1+\frac{P_i}{N}\right) \leq
\frac{1}{2}\log\left(1+\frac{1}{n}\sum_{i=1}^n \frac{P_i}{N}\right)
\,\!</math>

Because each codeword individually satisfies the power constraint, the average also satisfies the power constraint. Therefore,

:<math>
\frac{1}{n}\sum_{i=1}^n \frac{P_i}{N},
\,\!</math>

which we may apply to simplify the inequality above and get:
:<math>
\frac{1}{2}\log\left(1+\frac{1}{n}\sum_{i=1}^{n}\frac{P_i}{N}\right) \leq
\frac{1}{2}\log\left(1+\frac{P}{N}\right).
\,\!</math>

Therefore, it must be that <math>R \leq \frac{1}{2}\log \left(1+ \frac{P}{N}\right) + \varepsilon_n</math>. Therefore, ''R'' must be less than a value arbitrarily close to the capacity derived earlier, as <math>\varepsilon_n \rightarrow 0</math>.