Editing Channel capacity (section)

== Formal definition ==

The basic mathematical model for a communication system is the following:

:<math title="Channel model">\xrightarrow[\text{Message}]{W}
\begin{array}{|c|}\hline \text{Encoder} \\ f_n \\ \hline\end{array}
\xrightarrow[\mathrm{Encoded \atop sequence}]{X^n}
\begin{array}{|c|}\hline \text{Channel} \\ p(y|x) \\ \hline\end{array}
\xrightarrow[\mathrm{Received \atop sequence}]{Y^n}

\begin{array}{|c|}\hline \text{Decoder} \\ g_n \\ \hline\end{array}
\xrightarrow[\mathrm{Estimated \atop message}]{\hat W}

</math>

where:
* <math>W</math> is the message to be transmitted;
* <math>X</math> is the channel input symbol (<math>X^n</math> is a sequence of <math>n</math> symbols) taken in an [[Alphabet (formal languages)|alphabet]] <math>\mathcal{X}</math>;
* <math>Y</math> is the channel output symbol (<math>Y^n</math> is a sequence of <math>n</math> symbols) taken in an alphabet <math>\mathcal{Y}</math>;
* <math>\hat{W}</math> is the estimate of the transmitted message;
* <math>f_n</math> is the encoding function for a block of length <math>n</math>;
* <math>p(y|x) = p_{Y|X}(y|x)</math> is the noisy channel, which is modeled by a [[conditional probability distribution]]; and,
* <math>g_n</math> is the decoding function for a block of length <math>n</math>.

Let <math>X</math> and <math>Y</math> be modeled as random variables. Furthermore, let <math> p_{Y|X}(y|x)</math> be the [[conditional probability distribution]] function of <math>Y</math> given <math>X</math>, which is an inherent fixed property of the communication channel. Then the choice of the [[marginal distribution]] <math>p_X(x)</math> completely determines the [[Joint probability distribution|joint distribution]] <math>p_{X,Y}(x,y)</math> due to the identity

:<math>\ p_{X,Y}(x,y)=p_{Y|X}(y|x)\,p_X(x) </math>

which, in turn, induces a [[mutual information]] <math>I(X;Y)</math>. The '''channel capacity''' is defined as

:<math>\ C = \sup_{p_X(x)} I(X;Y)\, </math>

where the [[Infimum and supremum|supremum]] is taken over all possible choices of <math>p_X(x)</math>.