Editing Information content (section)

=== Categorical distribution ===
Generalizing all of the above cases, consider a [[Categorical variable|categorical]] [[discrete random variable]] with [[Support (mathematics)|support]] <math display="inline">\mathcal{S} = \bigl\{s_i\bigr\}_{i=1}^{N}</math> and [[probability mass function]] given by

<math display="block">p_X(k) = \begin{cases}
 p_i, & k = s_i \in \mathcal{S}
 \\ 0,  & \text{otherwise} .
\end{cases}</math>

For the purposes of information theory, the values <math>s \in \mathcal{S}</math> do not have to be [[number]]s; they can be any [[Mutually exclusive#Probability|mutually exclusive]] [[Event (probability theory)|events]] on a [[measure space]] of [[finite measure]] that has been [[Normalization (statistics)|normalized]] to a [[probability measure]] <math>p</math>.  [[Without loss of generality]], we can assume the categorical distribution is supported on the set <math display="inline">[N] = \left\{1, 2, \dots, N \right\}</math>; the mathematical structure is [[Isomorphism|isomorphic]] in terms of [[probability theory]] and therefore [[information theory]] as well.

The information of the outcome <math>X = x</math> is given

<math display="block">\operatorname{I}_X(x) = -\log_2{p_X(x)}.</math>

From these examples, it is possible to calculate the information of any set of [[Independent random variables|independent]] [[Discrete Random Variable|DRVs]] with known [[Probability distribution|distributions]] by [[Sigma additivity|additivity]].