Editing Shannon's source coding theorem (section)

=== Source coding theorem ===
In information theory, the source coding theorem (Shannon 1948)<ref name="Shannon"/> informally states that (MacKay 2003, pg. 81,<ref name="MacKay"/> Cover 2006, Chapter 5<ref name="Cover"/>):

<blockquote>{{mvar|N}} [[Independent and identically distributed random variables|i.i.d.]] random variables each with entropy {{math|''H''(''X'')}} can be compressed into more than {{math|''N&thinsp;H''(''X'')}} [[bit]]s with negligible risk of information loss, as {{math|''N'' → ∞}}; but conversely, if they are compressed into fewer than {{math|''N&thinsp;H''(''X'')}} bits it is virtually certain that information will be lost.</blockquote>The <math>NH(X)</math> coded sequence represents the compressed message in a biunivocal way, under the assumption that the decoder knows the source. From a practical point of view, this hypothesis is not always true. Consequently, when the entropy encoding is applied the transmitted message is <math>NH(X)+(inf. source)</math>. Usually, the information that characterizes the source is inserted at the beginning of the transmitted message.