Open main menu
Home
Random
Recent changes
Special pages
Community portal
Preferences
About Wikipedia
Disclaimers
Incubator escapee wiki
Search
User menu
Talk
Dark mode
Contributions
Create account
Log in
Editing
Information content
(section)
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
=== Additivity of independent events === The information content of two [[independent events]] is the sum of each event's information content. This property is known as [[Additive map|additivity]] in mathematics, and [[sigma additivity]] in particular in [[Measure (mathematics)|measure]] and probability theory. Consider two [[independent random variables]] <math display="inline">X,\, Y</math> with [[probability mass function]]s <math>p_X(x)</math> and <math>p_Y(y)</math> respectively. The [[joint probability mass function]] is <math display="block"> p_{X, Y}\!\left(x, y\right) = \Pr(X = x,\, Y = y) = p_X\!(x)\,p_Y\!(y) </math> because <math display="inline">X</math> and <math display="inline">Y</math> are [[Independence (probability theory)|independent]]. The information content of the [[Outcome (probability)|outcome]] <math> (X, Y) = (x, y)</math> is<math display="block"> \begin{align} \operatorname{I}_{X,Y}(x, y) &= -\log_2\left[p_{X,Y}(x, y)\right] = -\log_2 \left[p_X\!(x)p_Y\!(y)\right] \\[5pt] &= -\log_2 \left[p_X{(x)}\right] -\log_2 \left[p_Y{(y)}\right] \\[5pt] &= \operatorname{I}_X(x) + \operatorname{I}_Y(y) \end{align} </math> See ''{{Section link||Two independent, identically distributed dice|nopage=y}}'' below for an example. The corresponding property for [[likelihood]]s is that the [[log-likelihood]] of independent events is the sum of the log-likelihoods of each event. Interpreting log-likelihood as "support" or negative surprisal (the degree to which an event supports a given model: a model is supported by an event to the extent that the event is unsurprising, given the model), this states that independent events add support: the information that the two events together provide for statistical inference is the sum of their independent information.
Edit summary
(Briefly describe your changes)
By publishing changes, you agree to the
Terms of Use
, and you irrevocably agree to release your contribution under the
CC BY-SA 4.0 License
and the
GFDL
. You agree that a hyperlink or URL is sufficient attribution under the Creative Commons license.
Cancel
Editing help
(opens in new window)