Editing Helmholtz machine (section)

{{Short description|Type of artificial neural network}}
The '''Helmholtz machine''' (named after [[Hermann von Helmholtz]] and his concept of [[Helmholtz free energy]]) is a type of [[artificial neural network]] that can account for the hidden structure of a set of data by being trained to create a [[generative model]] of the original set of data.<ref name=“nc95“>{{Cite journal|title = The Helmholtz machine.|journal = Neural Computation|date = 1995|pages = 889–904|volume = 7|issue = 5|first1 = Dayan|last1 = Peter|authorlink1=Peter Dayan|first2 = Geoffrey E.|last2 = Hinton|authorlink2=Geoffrey Hinton|first3 = Radford M.|last3 = Neal|authorlink3=Radford M. Neal|first4 = Richard S.|last4 = Zemel|authorlink4=Richard Zemel|doi = 10.1162/neco.1995.7.5.889|pmid = 7584891|s2cid = 1890561|hdl = 21.11116/0000-0002-D6D3-E|hdl-access = free}} {{closed access}}</ref><ref>{{Cite journal |last=Hinton |first=Geoffrey E |last2=Zemel |first2=Richard |date=1993 |title=Autoencoders, Minimum Description Length and Helmholtz Free Energy |url=https://proceedings.neurips.cc/paper/1993/hash/9e3cfc48eccf81a0d57663e129aef3cb-Abstract.html |journal=Advances in Neural Information Processing Systems |publisher=Morgan-Kaufmann |volume=6}}</ref> The hope is that by learning economical [[Representation learning|representations]] of the data, the underlying structure of the generative model should reasonably approximate the hidden structure of the data set. A Helmholtz machine contains two networks, a bottom-up ''recognition'' network that takes the data as input and produces a distribution over hidden variables, and a top-down "generative" network that generates values of the hidden variables and the data itself. At the time, Helmholtz machines were one of a handful of learning architectures that used feedback as well as feedforward to ensure quality of learned models.<ref>Luttrell S.P. (1994). A Bayesian analysis of self-organizing maps. Neural Computation. 1994 Sep 1;6(5):767-94.[https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.948.8885&rep=rep1&type=pdf]</ref>

Helmholtz machines are usually trained using an [[unsupervised learning]] algorithm, such as the [[wake-sleep algorithm]].<ref>{{Cite journal|title = The wake-sleep algorithm for unsupervised neural networks|pmid= 7761831 |journal = Science|date = 1995-05-26|pages = 1158–1161|volume = 268|issue = 5214|doi = 10.1126/science.7761831|first1 = Geoffrey E.|last1 = Hinton|first2 = Peter|last2 = Dayan|authorlink2=Peter Dayan|first3 = Brendan J.|last3 = Frey|first4 = Radford|last4 = Neal|bibcode= 1995Sci...268.1158H |s2cid= 871473 }} {{closed access}}</ref> They are a precursor to [[variational autoencoder]]s, which are instead trained using [[backpropagation]]. Helmholtz machines  may also be used in applications requiring a supervised learning algorithm (e.g. character recognition, or position-invariant recognition of an object within a field).