Editing Information content (section)

==Relationship to entropy==
The [[Shannon entropy]] of the random variable <math>X </math> above is [[Shannon entropy#Definition|defined as]]
<math display="block">\begin{alignat}{2} 
 \Eta(X) &= \sum_{x} {-p_{X}{\left(x\right)} \log{p_{X}{\left(x\right)}}} \\
 &= \sum_{x} {p_{X}{\left(x\right)} \operatorname{I}_X(x)} \\
 &{\overset{\underset{\mathrm{def}}{}}{=}} \ 
  \operatorname{E}{\left[\operatorname{I}_X (X)\right]},
\end{alignat} </math>
by definition equal to the [[Expected value|expected]] information content of measurement of <math>X </math>.<ref>{{cite book|url=https://books.google.com/books?id=Lyte2yl1SPAC&pg=PA11|title=Fundamentals in Information Theory and Coding|author=Borda, Monica|publisher=Springer|year=2011|isbn=978-3-642-20346-6}}</ref>{{rp|11}}<ref>{{cite book|url=https://books.google.com/books?id=VpRESN24Zj0C&pg=PA19|title=Mathematics of Information and Coding|publisher=American Mathematical Society|year=2002|isbn=978-0-8218-4256-0|author1=Han, Te Sun |author2=Kobayashi, Kingo }}</ref>{{rp|19–20}}
The expectation is taken over the [[discrete random variable|discrete values]] over its [[Support (mathematics)|support]].  

Sometimes, the entropy itself is called the "self-information" of the random variable, possibly because the entropy satisfies <math>\Eta(X) = \operatorname{I}(X; X)</math>, where <math>\operatorname{I}(X;X)</math> is the [[mutual information]] of <math>X</math> with itself.<ref>Thomas M. Cover, Joy A. Thomas; Elements of Information Theory; p. 20; 1991.</ref>

For [[Continuous Random Variables|continuous random variables]] the corresponding concept is [[differential entropy]].