Editing Neural network (machine learning) (section)

== Types ==
<!-- Split to [[Types of artificial neural networks]] -->
{{Main|Types of artificial neural networks}}

ANNs have evolved into a broad family of techniques that have advanced the state of the art across multiple domains. The simplest types have one or more static components, including number of units, number of layers, unit weights and [[topology]]. Dynamic types allow one or more of these to evolve via learning. The latter is much more complicated but can shorten learning periods and produce better results. Some types allow/require learning to be "supervised" by the operator, while others operate independently. Some types operate purely in hardware, while others are purely software and run on general purpose computers.

Some of the main breakthroughs include: 

* Convolutional neural networks that have proven particularly successful in processing visual and other two-dimensional data;<ref>{{cite journal |vauthors=LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD |title=Backpropagation Applied to Handwritten Zip Code Recognition |journal=Neural Computation |volume=1 |issue=4 |pages=541–551 |date=1989 |doi=10.1162/neco.1989.1.4.541|s2cid=41312633 }}</ref><ref name="lecun2016slides">[[Yann LeCun]] (2016). Slides on Deep Learning [https://indico.cern.ch/event/510372/ Online] {{Webarchive|url=https://web.archive.org/web/20160423021403/https://indico.cern.ch/event/510372/ |date=23 April 2016 }}</ref> where long short-term memory avoids the [[vanishing gradient problem]]<ref name=":03">{{Cite journal |last1=Hochreiter|first1=Sepp|author-link=Sepp Hochreiter|last2=Schmidhuber|first2=Jürgen|s2cid=1915014|author-link2=Jürgen Schmidhuber|date=1 November 1997|title=Long Short-Term Memory|journal=Neural Computation|volume=9|issue=8 |pages=1735–1780 |doi=10.1162/neco.1997.9.8.1735|pmid=9377276|issn=0899-7667}}</ref> and can handle signals that have a mix of low and high frequency components aiding large-vocabulary speech recognition,<ref name="sak2014">{{Cite web|url=https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43905.pdf |title=Long Short-Term Memory recurrent neural network architectures for large scale acoustic modeling |last1=Sak|first1=Hasim |last2=Senior|first2=Andrew|date=2014|last3=Beaufays|first3=Francoise|archive-url=https://web.archive.org/web/20180424203806/https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43905.pdf|archive-date=24 April 2018}}</ref><ref name="liwu2015">{{cite arXiv|last1=Li|first1=Xiangang|last2=Wu|first2=Xihong|date=15 October 2014|title=Constructing Long Short-Term Memory based Deep Recurrent Neural Networks for Large Vocabulary Speech Recognition|eprint=1410.4281 |class=cs.CL}}</ref> [[Speech synthesis|text-to-speech synthesis]],<ref>{{Cite journal|title=TTS synthesis with bidirectional LSTM based Recurrent Neural Networks|pages=1964–1968|last1=Fan|first1=Y. |last2=Qian|first2=Y.|date=2014 |journal=Proceedings of the Annual Conference of the International Speech Communication Association, Interspeech|url=https://www.researchgate.net/publication/287741874|access-date=13 June 2017 |last3=Xie |first3=F.|last4=Soong|first4=F. K.}}</ref><ref name="scholarpedia2">{{cite journal |last1=Schmidhuber |first1=Jürgen |author-link=Jürgen Schmidhuber |year=2015 |title=Deep Learning |journal=Scholarpedia |volume=10 |issue=11 |pages=85–117 |bibcode=2015SchpJ..1032832S |doi=10.4249/scholarpedia.32832 |doi-access=free}}</ref><ref name="zen2015">{{Cite web|url=https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43266.pdf|title=Unidirectional Long Short-Term Memory Recurrent Neural Network with Recurrent Output Layer for Low-Latency Speech Synthesis|last1=Zen|first1=Heiga|last2=Sak|first2=Hasim|date=2015|website=Google.com|publisher=ICASSP|pages=4470–4474|access-date=27 June 2017|archive-date=9 May 2021|archive-url=https://web.archive.org/web/20210509123113/https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43266.pdf|url-status=live}}</ref> and photo-real talking heads;<ref name="fan2015">{{Cite journal|last1=Fan|first1=Bo|last2=Wang|first2=Lijuan|last3=Soong|first3=Frank K.|last4=Xie|first4=Lei|date=2015|title=Photo-Real Talking Head with Deep Bidirectional LSTM|url=https://www.microsoft.com/en-us/research/wp-content/uploads/2015/04/icassp2015_fanbo_1009.pdf|journal=Proceedings of ICASSP|access-date=27 June 2017|archive-date=1 November 2017|archive-url=https://web.archive.org/web/20171101052317/https://www.microsoft.com/en-us/research/wp-content/uploads/2015/04/icassp2015_fanbo_1009.pdf|url-status=live}}</ref> 
* Competitive networks such as [[generative adversarial network]]s in which multiple networks (of varying structure) compete with each other, on tasks such as winning a game<ref name="preprint">{{Cite arXiv |eprint=1712.01815|class=cs.AI|first1=David|last1=Silver|first2=Thomas|last2=Hubert|author-link1=David Silver (programmer)|title=Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm|date=5 December 2017|first3=Julian|last3=Schrittwieser|first4=Ioannis|last4=Antonoglou |first5=Matthew|last5=Lai |first6=Arthur|last6=Guez|first7=Marc|last7=Lanctot|first8=Laurent|last8=Sifre |first9=Dharshan|last9=Kumaran|author-link9=Dharshan Kumaran|first10=Thore|last10=Graepel|first11=Timothy |last11=Lillicrap|first12=Karen |last12=Simonyan|first13=Demis|last13=Hassabis|author-link13=Demis Hassabis}}</ref> or on deceiving the opponent about the authenticity of an input.<ref name="GANnips"/>