Information geometry

The set of all normal distributions forms a statistical manifold with hyperbolic geometry.

Information geometry is an interdisciplinary field that applies the techniques of differential geometry to study probability theory and statistics.<ref>Template:Cite journal</ref> It studies statistical manifolds, which are Riemannian manifolds whose points correspond to probability distributions.

IntroductionEdit

{{ safesubst:#invoke:Unsubst||date=__DATE__ |$B= Template:Ambox }}

Historically, information geometry can be traced back to the work of C. R. Rao, who was the first to treat the Fisher matrix as a Riemannian metric.<ref>Template:Cite journal Reprinted in Template:Cite book</ref><ref>Template:Cite book</ref> The modern theory is largely due to Shun'ichi Amari, whose work has been greatly influential on the development of the field.<ref>Template:Cite journal</ref>

Classically, information geometry considered a parametrized statistical model as a Riemannian, conjugate connection, statistical, and dually flat manifolds. Unlike usual smooth manifolds with tensor metric and Levi-Civita connection, these take into account conjugate connection, torsion, and Amari-Chentsov metric.<ref>Template:Cite journal</ref> All presented above geometric structures find application in information theory and machine learning. For such models, there is a natural choice of Riemannian metric, known as the Fisher information metric. In the special case that the statistical model is an exponential family, it is possible to induce the statistical manifold with a Hessian metric (i.e a Riemannian metric given by the potential of a convex function). In this case, the manifold naturally inherits two flat affine connections, as well as a canonical Bregman divergence. Historically, much of the work was devoted to studying the associated geometry of these examples. In the modern setting, information geometry applies to a much wider context, including non-exponential families, nonparametric statistics, and even abstract statistical manifolds not induced from a known statistical model. The results combine techniques from information theory, affine differential geometry, convex analysis and many other fields. One of the most perspective information geometry approaches find applications in machine learning. For example, the developing of information-geometric optimization methods (mirror descent<ref>Template:Cite journal</ref> and natural gradient descent<ref>Template:Cite journal</ref>).

The standard references in the field are Shun’ichi Amari and Hiroshi Nagaoka's book, Methods of Information Geometry,<ref>Template:Cite book</ref> and the more recent book by Nihat Ay and others.<ref>Template:Cite book</ref> A gentle introduction is given in the survey by Frank Nielsen.<ref>Template:Cite journal</ref> In 2018, the journal Information Geometry was released, which is devoted to the field.

ContributorsEdit

{{ safesubst:#invoke:Unsubst||date=__DATE__ |$B= Template:Ambox }} The history of information geometry is associated with the discoveries of at least the following people, and many others. Template:Div col