Editing Unsupervised learning (section)

=== Networks ===
This table shows connection diagrams of various unsupervised networks, the details of which will be given in the section Comparison of Networks.  Circles are neurons and edges between them are connection weights.  As network design changes, features are added on to enable new capabilities or removed to make learning faster.  For instance, neurons change between deterministic (Hopfield) and stochastic (Boltzmann) to allow robust output, weights are removed within a layer (RBM) to hasten learning, or connections are allowed to become asymmetric (Helmholtz).

{| class="wikitable"
|-
! [[Hopfield network|Hopfield]] !! [[Boltzmann machine|Boltzmann]] !! [[Restricted Boltzmann machine|RBM]] !! [[Stacked Restricted Boltzmann Machine|Stacked Boltzmann]]
|-
| [[File:Hopfield-net-vector.svg          |thumb|A network based on magnetic domains in iron with a single self-connected layer.  It can be used as a content addressable memory.]] 
|| [[File:Boltzmannexamplev1.png          |thumb|Network is separated into 2 layers (hidden vs. visible), but still using symmetric 2-way weights.  Following Boltzmann's thermodynamics, individual  probabilities give rise to macroscopic energies.]] 
|| [[File:Restricted Boltzmann machine.svg|thumb|Restricted Boltzmann Machine.  This is a Boltzmann machine where lateral connections within a layer are prohibited to make analysis tractable.]] 
|| [[File:Stacked-boltzmann.png|thumb|This network has multiple RBM's to encode a hierarchy of hidden features.  After a single RBM is trained, another blue hidden layer (see left RBM) is added, and the top 2 layers are trained as a red & blue RBM.  Thus the middle layers of an RBM acts as hidden or visible, depending on the training phase it is in.]]
|}

{| class="wikitable"
|-
! [[Helmholtz machine|Helmholtz]] !! [[Autoencoder]] !! [[Variational autoencoder|VAE]]
|-
|| [[File:Helmholtz Machine.png           |thumb|Instead of the bidirectional symmetric connection of the stacked Boltzmann machines, we have separate one-way connections to form a loop.  It does both generation and discrimination.]] 
|| [[File:Autoencoder_schema.png          |thumb|A feed forward network that aims to find a good middle layer representation of its input world.  This network is deterministic, so it is not as robust as its successor the VAE.]] 
|| [[File:VAE blocks.png                  |thumb|Applies Variational Inference to the Autoencoder.  The middle layer is a set of means & variances for Gaussian distributions.  The stochastic nature allows for more robust imagination than the deterministic autoencoder.]]
|}

Of the networks bearing people's names, only Hopfield worked directly with neural networks.  Boltzmann and Helmholtz came before artificial neural networks, but their work in physics and physiology inspired the analytical methods that were used.