Editing Entropy (information theory) (section)

===Alternative characterization via additivity and subadditivity===

Another succinct axiomatic characterization of Shannon entropy was given by [[János Aczél (mathematician)|Aczél]], Forte and Ng,<ref name="aczelentropy">{{cite journal|last1=Aczél|first1=J.|title=Why the Shannon and Hartley entropies are 'natural'|last2=Forte|first2=B.|last3=Ng|first3=C. T.|journal=Advances in Applied Probability|date=1974|volume=6|issue=1|pages=131–146|doi=10.2307/1426210 |jstor=1426210 |s2cid=204177762 }}</ref> via the following properties:

# Subadditivity:  <math>\Eta(X,Y) \le \Eta(X)+\Eta(Y)</math>   for jointly distributed random variables <math>X,Y</math>.
# Additivity:  <math>\Eta(X,Y) = \Eta(X)+\Eta(Y)</math> when the random variables <math>X,Y</math> are independent.
# Expansibility:  <math>\Eta_{n+1}(p_1, \ldots, p_n, 0) = \Eta_n(p_1, \ldots, p_n)</math>, i.e., adding an outcome with probability zero does not change the entropy.
# Symmetry: <math>\Eta_n(p_1, \ldots, p_n)</math> is invariant under permutation of <math>p_1, \ldots, p_n</math>.
# Small for small probabilities:  <math>\lim_{q \to 0^+} \Eta_2(1-q, q) = 0</math>.

==== Discussion ====
It was shown that any function <math>\Eta</math> satisfying the above properties must be a constant multiple of Shannon entropy, with a non-negative constant.<ref name="aczelentropy"/> Compared to the previously mentioned characterizations of entropy, this characterization focuses on the properties of entropy as a function of random variables (subadditivity and additivity), rather than the properties of entropy as a function of the probability vector <math>p_1,\ldots ,p_n</math>.

It is worth noting that if we drop the "small for small probabilities" property, then <math>\Eta</math> must be a non-negative linear combination of the Shannon entropy and the [[Hartley entropy]].<ref name="aczelentropy"/>