Editing Entropy (information theory) (section)

=== Measure theory ===

Entropy can be formally defined in the language of [[measure theory]] as follows:<ref>{{nlab|id=entropy|title=Entropy}}</ref> Let <math>(X, \Sigma, \mu)</math> be a [[probability space]]. Let <math>A \in \Sigma</math> be an [[event (probability theory)|event]]. The [[surprisal]] of <math>A</math> is
<math display="block"> \sigma_\mu(A) = -\ln \mu(A) .</math>

The ''expected'' surprisal of <math>A</math> is
<math display="block"> h_\mu(A) = \mu(A) \sigma_\mu(A) .</math>

A <math>\mu</math>-almost [[partition of a set|partition]] is a [[set family]] <math>P \subseteq \mathcal{P}(X)</math> such that <math>\mu(\mathop{\cup} P) = 1</math> and <math>\mu(A \cap B) = 0</math> for all distinct <math>A, B \in P</math>. (This is a relaxation of the usual conditions for a partition.) The entropy of <math>P</math> is
<math display="block">  \Eta_\mu(P) = \sum_{A \in P} h_\mu(A) .</math>

Let <math>M</math> be a [[sigma-algebra]] on <math>X</math>. The entropy of <math>M</math> is
<math display="block"> \Eta_\mu(M) = \sup_{P \subseteq M} \Eta_\mu(P) .</math>
Finally, the entropy of the probability space is <math>\Eta_\mu(\Sigma)</math>, that is, the entropy with respect to <math>\mu</math> of the sigma-algebra of ''all'' measurable subsets of <math>X</math>.

Recent studies on layered dynamical systems have introduced the concept of symbolic conditional entropy, further extending classical entropy measures to more abstract informational structures.<ref>{{cite web |last=Alpay |first=F. |year=2025 |title=Symbolic Conditional Entropy in Layered Dynamical Systems |publisher=Zenodo |url=https://doi.org/10.5281/zenodo.15354902 |doi=10.5281/zenodo.15354902 |access-date=7 May 2025}}</ref>