Editing Boltzmann machine (section)

{{short description|Type of stochastic recurrent neural network}}
[[File:Boltzmannexamplev1.png|thumb|alt=A graphical representation of an example Boltzmann machine.| A graphical representation of an example Boltzmann machine. Each undirected edge represents dependency. In this example there are 3 hidden units and 4 visible units. This is not a restricted Boltzmann machine.]]
A '''Boltzmann machine''' (also called '''Sherrington–Kirkpatrick model with external field''' or '''stochastic Ising model'''), named after [[Ludwig Boltzmann]], is a [[spin glass|spin-glass]] model with an external field, i.e., a [[Spin glass#Sherrington–Kirkpatrick model|Sherrington–Kirkpatrick model]],<ref>{{citation|title=Solvable Model of a Spin-Glass|number=35|year=1975|author1= Sherrington, David|author2=Kirkpatrick, Scott|journal=Physical Review Letters|volume=35|pages=1792–1796|doi=10.1103/PhysRevLett.35.1792|bibcode=1975PhRvL..35.1792S}}</ref> that is a stochastic [[Ising model]]. It is a [[statistical physics]] technique applied in the context of [[cognitive science]].<ref name=":0">{{cite journal |last=Ackley |first=David H. |author2=Hinton, Geoffrey E. |author3=Sejnowski, Terrence J. |year=1985 |title=A Learning Algorithm for Boltzmann Machines |url=http://learning.cs.toronto.edu/~hinton/absps/cogscibm.pdf |journal=[[Cognitive Science (journal)|Cognitive Science]] |volume=9 |issue=1 |pages=147–169 |doi=10.1207/s15516709cog0901_7 |archive-url=https://web.archive.org/web/20110718022336/http://learning.cs.toronto.edu/~hinton/absps/cogscibm.pdf |archive-date=18 July 2011 |doi-access=free}}</ref> It is also classified as a [[Markov random field]].<ref>{{Cite journal|last=Hinton|first=Geoffrey E.|date=2007-05-24|title=Boltzmann machine|journal=Scholarpedia|language=en|volume=2|issue=5|page=1668|doi=10.4249/scholarpedia.1668|bibcode=2007SchpJ...2.1668H|issn=1941-6016|doi-access=free}}</ref>

Boltzmann machines are theoretically intriguing because of the locality and [[Hebbian]] nature of their training algorithm (being trained by Hebb's rule), and because of their [[Parallelism (computing)|parallelism]] and the resemblance of their dynamics to simple [[physical process]]es.  Boltzmann machines with unconstrained connectivity have not been proven useful for practical problems in [[machine learning]] or [[inference]], but if the connectivity is properly constrained, the learning can be made efficient enough to be useful for practical problems.<ref>{{cite book|title=International Neural Network Conference|first=Thomas R.|last=Osborn|date=1 January 1990|publisher=Springer Netherlands|pages=[https://archive.org/details/innc90parisinter0001inte/page/785 785]|doi=10.1007/978-94-009-0643-3_76|chapter=Fast Teaching of Boltzmann Machines with Local Inhibition|isbn=978-0-7923-0831-7|chapter-url=https://archive.org/details/innc90parisinter0001inte/page/785}}</ref>

They are named after the [[Boltzmann distribution]] in [[statistical mechanics]], which is used in their [[sampling function]].  They were heavily popularized and promoted by [[Geoffrey Hinton]], [[Terry Sejnowski]] and [[Yann LeCun]] in cognitive sciences communities, particularly in [[machine learning]],<ref name=":0" />  as part of "[[energy-based model]]s" (EBM), because [[Hamiltonian function|Hamiltonians]] of [[spin glasses]] as energy are used as a starting point to define the learning task.<ref>{{citation|title=On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models|number=34|year=2020|author1=Nijkamp, E. |author2=Hill, M. E|author3= Han, T. |journal=Proceedings of the AAAI Conference on Artificial Intelligence|volume=4|pages=5272–5280|doi=10.1609/aaai.v34i04.5973|url=https://ojs.aaai.org/index.php/AAAI/article/view/5973|doi-access=free|arxiv=1903.12370}}</ref>