Editing Information content (section)

=== Two independent, identically distributed dice ===
Suppose we have two [[Independent and identically distributed random variables|independent, identically distributed random variables]] <math display="inline">X,\, Y \sim \mathrm{DU}[1, 6]</math> each corresponding to an [[Independent random variables|independent]] fair 6-sided dice roll.  The [[Joint probability distribution|joint distribution]] of <math>X</math> and <math>Y</math> is<math display="block"> \begin{align}
 p_{X, Y}\!\left(x, y\right) & {} = \Pr(X = x,\, Y = y) 
 = p_X\!(x)\,p_Y\!(y) \\
 & {} = \begin{cases}
  \displaystyle{1 \over 36}, \ &x, y \in [1, 6] \cap \mathbb{N} \\
  0 & \text{otherwise.} \end{cases}
\end{align}</math>

The information content of the [[random variate]] <math> (X, Y) = (2,\, 4)</math> is
<math display="block"> \begin{align}
\operatorname{I}_{X, Y}{(2, 4)} 
 &= -\log_2\!{\left[p_{X,Y}{(2, 4)}\right]}
 = \log_2\!{36} = 2 \log_2\!{6} \\
 & \approx 5.169925 \text{ Sh},
\end{align}
</math>
and can also be calculated by [[#Additivity of independent events|additivity of events]]
<math display="block"> \begin{align}
\operatorname{I}_{X, Y}{(2, 4)} 
 &= -\log_2\!{\left[p_{X,Y}{(2, 4)}\right]}
 = -\log_2\!{\left[p_X(2)\right]} -\log_2\!{\left[p_Y(4)\right]} \\
 & = 2\log_2\!{6} \\
 & \approx 5.169925 \text{ Sh}.
\end{align}
</math>

==== Information from frequency of rolls ====
If we receive information about the value of the dice [[Twelvefold way#case fx|without knowledge]] of which die had which value, we can formalize the approach with so-called counting variables
<math display="block"> C_k := \delta_k(X) + \delta_k(Y) = \begin{cases} 
 0, & \neg\, (X = k \vee Y = k) \\
 1, & \quad X = k\, \veebar \, Y = k \\
 2, & \quad X = k\, \wedge \, Y = k
\end{cases}
</math>
for <math> k \in \{1, 2, 3, 4, 5, 6\}</math>, then <math display="inline"> \sum_{k=1}^{6}{C_k} = 2</math> and the counts have the [[multinomial distribution]]
<math display="block"> \begin{align}
 f(c_1,\ldots,c_6) & {} = \Pr(C_1 = c_1 \text{ and } \dots \text{ and } C_6 = c_6) \\
 & {} = \begin{cases} { \displaystyle {1\over{18}}{1 \over c_1!\cdots c_k!}}, 
    \ & \text{when } \sum_{i=1}^6 c_i=2 \\
  0 & \text{otherwise,} \end{cases} \\
 & {} = \begin{cases} {1 \over 18}, 
  \ & \text{when 2 } c_k \text{ are } 1 \\
  {1 \over 36}, \ & \text{when exactly one } c_k = 2 \\
  0, \ & \text{otherwise.}
 \end{cases}
\end{align}</math>

To verify this, the 6 outcomes <math display="inline">(X, Y) \in \left\{(k, k)\right\}_{k = 1}^{6} = \left\{
 (1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6)
\right\}</math> correspond to the event <math>C_k = 2</math> and a [[total probability]] of {{Sfrac|6}}.  These are the only events that are faithfully preserved with identity of which dice rolled which outcome because the outcomes are the same.  Without knowledge to distinguish the dice rolling the other numbers, the other <math display="inline"> \binom{6}{2} = 15</math> [[combination]]s correspond to one die rolling one number and the other die rolling a different number, each having probability {{Sfrac|18}}. Indeed, <math display="inline"> 6 \cdot \tfrac{1}{36} + 15 \cdot \tfrac{1}{18} = 1</math>, as required.

Unsurprisingly, the information content of learning that both dice were rolled as the same particular number is more than the information content of learning that one dice was one number and the other was a different number.  Take for examples the events <math> A_k = \{(X, Y) = (k, k)\}</math> and <math> B_{j, k} = \{c_j = 1\} \cap \{c_k = 1\}</math> for <math> j \ne k, 1 \leq j, k \leq 6</math>. For example, <math> A_2 = \{X = 2 \text{ and } Y = 2\}</math> and <math> B_{3, 4} = \{(3, 4), (4, 3)\}</math>.

The information contents are
<math display="block"> \operatorname{I}(A_2) = -\log_2\!{\tfrac{1}{36}} = 5.169925 \text{ Sh}</math>
<math display="block"> \operatorname{I}\left(B_{3, 4}\right) = - \log_2 \! \tfrac{1}{18} = 4.169925 \text{ Sh}</math>

Let <math display="inline"> \text{Same} = \bigcup_{i = 1}^{6}{A_i}</math> be the event that both dice rolled the same value and <math> \text{Diff} = \overline{\text{Same}}</math> be the event that the dice differed. Then <math display="inline"> \Pr(\text{Same}) = \tfrac{1}{6}</math> and <math display="inline"> \Pr(\text{Diff}) = \tfrac{5}{6}</math>. The information contents of the events are
<math display="block"> \operatorname{I}(\text{Same}) = -\log_2\!{\tfrac{1}{6}} = 2.5849625 \text{ Sh}</math>
<math display="block"> \operatorname{I}(\text{Diff}) = -\log_2\!{\tfrac{5}{6}} = 0.2630344 \text{ Sh}.</math>

==== Information from sum of dice ====

The probability mass or density function (collectively [[probability measure]]) of the [[Sum of independent random variables|sum of two independent random variables]] [[Convolution#Convolution of measures|is the convolution of each probability measure]].  In the case of independent fair 6-sided dice rolls, the random variable <math> Z = X + Y</math> has probability mass function <math display="inline"> p_Z(z) = p_X(x) * p_Y(y) = {6 - |z - 7| \over 36} </math>, where <math> *</math> represents the [[discrete convolution]].  The [[Outcome (probability)|outcome]] <math> Z = 5 </math> has probability <math display="inline"> p_Z(5) = \frac{4}{36} = {1 \over 9} </math>. Therefore, the information asserted is<math display="block"> \operatorname{I}_Z(5) = -\log_2{\tfrac{1}{9}} = \log_2{9}
 \approx 3.169925 \text{ Sh}. 
</math>