Editing Connectionism (section)

{{Short description|Cognitive science approach}}
[[File:Artificial_neural_network.svg|thumb|A 'second wave' connectionist (ANN) model with a hidden layer]]
'''Connectionism''' is an approach to the study of human mental processes and cognition that utilizes mathematical models known as connectionist networks or [[artificial neural network]]s.<ref>{{Cite web|url=https://iep.utm.edu/connectionism-cognition/#:~:text=Connectionism%20is%20an%20approach%20to,%2C%20neuron%2Dlike%20processing%20units.|title=Internet Encyclopedia of Philosophy|website=iep.utm.edu|access-date=2023-08-19}}</ref>

Connectionism has had many "waves" since its beginnings. The first wave appeared 1943 with [[Warren Sturgis McCulloch]] and [[Walter Pitts]] both focusing on comprehending neural circuitry through a formal and mathematical approach,<ref>{{Cite journal |last1=McCulloch |first1=Warren S. |last2=Pitts |first2=Walter |date=1943-12-01 |title=A logical calculus of the ideas immanent in nervous activity |url=https://doi.org/10.1007/BF02478259 |journal=The Bulletin of Mathematical Biophysics |language=en |volume=5 |issue=4 |pages=115–133 |doi=10.1007/BF02478259 |issn=1522-9602|url-access=subscription }}</ref> and [[Frank Rosenblatt]] who published the 1958 paper "The Perceptron: A Probabilistic Model For Information Storage and Organization in the Brain" in ''Psychological Review'', while working at the Cornell Aeronautical Laboratory.<ref name="2019TheCuriousCaseOfConnectionism">{{Cite journal|last=Berkeley |first= Istvan S. N.|date=2019|title=The Curious Case of Connectionism |journal=Open Philosophy |volume=2019 |issue=2 |pages=190–205|doi= 10.1515/opphil-2019-0018|s2cid= 201061823|doi-access=free }}</ref>
The first wave ended with the 1969 book about the limitations of the original perceptron idea, written by [[Marvin Minsky]] and [[Seymour Papert]], which contributed to discouraging major funding agencies in the US from investing in connectionist research.<ref name="2006_Margaret_Boden_book">{{Cite book |last=Boden |first=Margaret |url=https://archive.org/details/mindasmachinehis0002bode/page/n5/mode/thumb |title=Mind as Machine: A History of Cognitive Science |publisher=Oxford U.P |year=2006 |isbn=978-0-262-63268-3 |location=Oxford |pages=914 |url-access=limited}}</ref> With a few noteworthy deviations, most connectionist research entered a period of inactivity until the mid-1980s. The term ''connectionist model'' was reintroduced in a 1982 paper in the journal ''Cognitive Science'' by Jerome Feldman and Dana Ballard.

The second wave blossomed in the late 1980s, following a 1987 book about Parallel Distributed Processing by [[James L. McClelland]], [[David E. Rumelhart]] et al., which introduced a couple of improvements to the simple perceptron idea, such as intermediate processors (now known as "[[hidden layers]]") alongside input and output units, and used a [[Sigmoid function|sigmoid]] [[activation function]] instead of the old "all-or-nothing" function. Their work built upon that of [[John Hopfield]], who was a key figure investigating the mathematical characteristics of sigmoid activation functions.<ref name="2019TheCuriousCaseOfConnectionism"/> From the late 1980s to the mid-1990s, connectionism took on an almost revolutionary tone when Schneider,<ref name="1987_Paradigm_Shift_in_Psychology">{{Cite journal|last=Schneider |first=Walter |date=1987 |title=Connectionism: Is it a Paradigm Shift for Psychology? |journal=Behavior Research Methods, Instruments, & Computers |volume=19 |pages=73–83|doi=10.1515/opphil-2019-0018 |s2cid=201061823 |doi-access=free }}</ref> [[Terence Horgan]] and Tienson posed the question of whether connectionism represented a [[paradigm shift|fundamental shift]] in psychology and so-called "good old-fashioned AI," or [[GOFAI]].<ref name="2019TheCuriousCaseOfConnectionism"/> Some advantages of the second wave connectionist approach included its applicability to a broad array of functions, structural approximation to biological neurons, low requirements for innate structure, and capacity for [[graceful degradation]].<ref name=":2">{{Cite book|title=The Algebraic Mind: Integrating Connectionism and Cognitive Science (Learning, Development, and Conceptual Change)|url=https://archive.org/details/algebraicmindint00marc_403|url-access=limited|last=Marcus|first=Gary F.|publisher=MIT Press|year=2001|isbn=978-0-262-63268-3|location=Cambridge, Massachusetts|pages=[https://archive.org/details/algebraicmindint00marc_403/page/n43 27]–28}}</ref> Its disadvantages included the difficulty in deciphering how ANNs process information or account for the compositionality of mental representations, and a resultant difficulty explaining phenomena at a higher level.<ref name=":1">{{Cite journal|last=Smolensky|first=Paul|date=1999|title=Grammar-based Connectionist Approaches to Language|journal=Cognitive Science|volume=23|issue=4|pages=589–613|doi=10.1207/s15516709cog2304_9|doi-access=free}}</ref>

The current (third) wave has been marked by advances in [[deep learning]], which have made possible the creation of [[large language model]]s.<ref name="2019TheCuriousCaseOfConnectionism"/> The success of deep-learning networks in the past decade has greatly increased the popularity of this approach, but the complexity and scale of such networks has brought with them increased [[Explainable artificial intelligence|interpretability problems]].<ref name=":0">{{cite book|url=https://plato.stanford.edu/archives/fall2018/entries/connectionism/|title=The Stanford Encyclopedia of Philosophy|first=James|last=Garson|editor-first=Edward N.|editor-last=Zalta|date=27 November 2018|publisher=Metaphysics Research Lab, Stanford University|via=Stanford Encyclopedia of Philosophy}}</ref>