Editing Artificial neuron (section)

==History==
The first artificial neuron was the Threshold Logic Unit (TLU), or Linear Threshold Unit,<ref name="Anthony2001">{{cite book|author=Martin Anthony|title=Discrete Mathematics of Neural Networks: Selected Topics|url=https://books.google.com/books?id=qOy4yLBqhFcC&pg=PA3|date=January 2001|publisher=SIAM|isbn=978-0-89871-480-7|pages=3–}}</ref> first proposed by [[Warren McCulloch]] and [[Walter Pitts]] in 1943 in ''[[A Logical Calculus of the Ideas Immanent in Nervous Activity|A logical calculus of the ideas immanent in nervous activity]]''. The model was specifically targeted as a computational model of the "nerve net" in the brain.<ref name="Aggarwal2014">{{cite book|author=Charu C. Aggarwal|title=Data Classification: Algorithms and Applications|url=https://books.google.com/books?id=gJhBBAAAQBAJ&pg=PA209|date=25 July 2014|publisher=CRC Press|isbn=978-1-4665-8674-1|pages=209–}}</ref> As an activation function, it employed a threshold, equivalent to using the [[Heaviside step function]]. Initially, only a simple model was considered, with binary inputs and outputs, some restrictions on the possible weights, and a more flexible threshold value. Since the beginning it was already noticed that any [[Boolean function]] could be implemented by networks of such devices, what is easily seen from the fact that one can implement the AND and OR functions, and use them in the [[disjunctive normal form|disjunctive]] or the [[conjunctive normal form]].
Researchers also soon realized that cyclic networks, with [[feedback]]s through neurons, could define dynamical systems with memory, but most of the research concentrated (and still does) on strictly [[feed-forward network]]s because of the smaller difficulty they present.

One important and pioneering artificial neural network that used the linear threshold function was the [[perceptron]], developed by [[Frank Rosenblatt]]. This model already considered more flexible weight values in the neurons, and was used in machines with adaptive capabilities. The representation of the threshold values as a bias term was introduced by [[Bernard Widrow]] in 1960 – see [[ADALINE]].

In the late 1980s, when research on neural networks regained strength, neurons with more continuous shapes started to be considered. The possibility of differentiating the activation function allows the direct use of the [[gradient descent]] and other optimization algorithms for the adjustment of the weights. Neural networks also started to be used as a general [[function approximation]] model. The best known training algorithm called [[backpropagation]] has been rediscovered several times but its first development goes back to the work of [[Paul Werbos]].<ref>[[Paul Werbos]], Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD thesis, Harvard University, 1974</ref><ref>{{cite journal | last=Werbos | first=P.J. |author-link=Paul Werbos| title=Backpropagation through time: what it does and how to do it | journal=Proceedings of the IEEE | volume=78 | issue=10 | year=1990 | issn=0018-9219 | doi=10.1109/5.58337 | pages=1550–1560| s2cid=18470994 | url=https://zenodo.org/record/1262035 }}</ref>