Editing Connectionism (section)

==Connectionism vs. computationalism debate==
As connectionism became increasingly popular in the late 1980s, some researchers (including [[Jerry Fodor]], [[Steven Pinker]] and others) reacted against it.  They argued that connectionism, as then developing, threatened to obliterate what they saw as the progress being made in the fields of cognitive science and psychology by the classical approach of [[computationalism]]. Computationalism is a specific form of cognitivism that argues that mental activity is [[Computing|computational]], that is, that the mind operates by performing purely formal operations on symbols, like a [[Turing machine]].  Some researchers argued that the trend in connectionism represented a reversion toward [[associationism]] and the abandonment of the idea of a [[language of thought]], something they saw as mistaken. In contrast, those very tendencies made connectionism attractive for other researchers.

Connectionism and computationalism need not be at odds, but the debate in the late 1980s and early 1990s led to opposition between the two approaches.  Throughout the debate, some researchers have argued that connectionism and computationalism are fully compatible, though full consensus on this issue has not been reached.  Differences between the two approaches include the following:

* Computationalists posit symbolic models that are structurally similar to underlying brain structure, whereas connectionists engage in "low-level" modeling, trying to ensure that their models resemble neurological structures.
* Computationalists in general focus on the structure of explicit symbols ([[mental models]]) and [[syntactical]] rules for their internal manipulation, whereas connectionists focus on learning from environmental stimuli and storing this information in a form of connections between neurons.
* Computationalists believe that internal mental activity consists of manipulation of explicit symbols, whereas connectionists believe that the manipulation of explicit symbols provides a poor model of mental activity.
* Computationalists often posit [[domain specificity|domain specific]] symbolic sub-systems designed to support learning in specific areas of cognition (e.g., language, intentionality, number), whereas connectionists posit one or a small set of very general learning-mechanisms.

Despite these differences, some theorists have proposed that the connectionist architecture is simply the manner in which organic brains happen to implement the symbol-manipulation system. This is logically possible, as it is well known that connectionist models can implement symbol-manipulation systems of the kind used in computationalist models,<ref name=":3">{{Cite journal|last=Chang|first=Franklin|date=2002|title=Symbolically speaking: a connectionist model of sentence production|journal=Cognitive Science|language=en|volume=26|issue=5|pages=609–651|doi=10.1207/s15516709cog2605_3|issn=1551-6709|doi-access=free}}</ref> as indeed they must be able if they are to explain the human ability to perform symbol-manipulation tasks. Several cognitive models combining both symbol-manipulative and connectionist architectures have been proposed. Among them are [[Paul Smolensky]]'s Integrated Connectionist/Symbolic Cognitive Architecture (ICS).<ref name=":0" /><ref>{{Cite journal|last=Smolensky|first=Paul|date=1990|title=Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems|url=http://www.lscp.net/persons/dupoux/teaching/AT1_2012/papers/Smolensky_1990_TensorProductVariableBinding.AI.pdf|journal=Artificial Intelligence|volume=46|issue=1–2|pages=159–216|doi=10.1016/0004-3702(90)90007-M}}</ref> and [[Ron Sun]]'s [[CLARION (cognitive architecture)]].  But the debate rests on whether this symbol manipulation forms the foundation of cognition in general, so this is not a potential vindication of computationalism. Nonetheless, computational descriptions may be helpful high-level descriptions of cognition of logic, for example.

The debate was largely centred on logical arguments about whether connectionist networks could produce the syntactic structure observed in this sort of reasoning. This was later achieved although using fast-variable binding abilities outside of those standardly assumed in connectionist models.<ref name=":3" /><ref>{{Cite journal|last1=Shastri|first1=Lokendra|last2=Ajjanagadde|first2=Venkat|date=September 1993|title=From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony|journal=Behavioral and Brain Sciences|language=en|volume=16|issue=3|pages=417–451|doi=10.1017/S0140525X00030910|s2cid=14973656|issn=1469-1825}}</ref>

Part of the appeal of computational descriptions is that they are relatively easy to interpret, and thus may be seen as contributing to our understanding of particular mental processes, whereas connectionist models are in general more opaque, to the extent that they may be describable only in very general terms (such as specifying the learning algorithm, the number of units, etc.), or in unhelpfully low-level terms. In this sense, connectionist models may instantiate, and thereby provide evidence for, a broad theory of cognition (i.e., connectionism), without representing a helpful theory of the particular process that is being modelled. In this sense, the debate might be considered as to some extent reflecting a mere difference in the level of analysis in which particular theories are framed. Some researchers suggest that the analysis gap is the consequence of connectionist mechanisms giving rise to [[Emergence|emergent phenomena]] that may be describable in computational terms.<ref>{{Cite journal|last=Ellis|first=Nick C.|date=1998|title=Emergentism, Connectionism and Language Learning|url=http://www-personal.umich.edu/~ncellis/NickEllis/Publications_files/Emergentism.pdf|journal=Language Learning|volume=48|issue=4|pages=631–664|doi=10.1111/0023-8333.00063}}</ref>

In the 2000s, the popularity of [[Cognitive Model#Dynamical systems|dynamical systems]] in [[philosophy of mind]] have added a new perspective on the debate;<ref>{{Citation |last=Van Gelder |first=Tim |year=1998 |title=The dynamical hypothesis in cognitive science |journal=Behavioral and Brain Sciences |volume=21 |issue=5 |pages=615–28; discussion 629–65 |doi=10.1017/S0140525X98001733 |pmid=10097022 |url= https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S0140525X98271731 |access-date=28 May 2022|url-access=subscription }}</ref><ref>{{Cite journal |last=Beer |first=Randall D. |date=March 2000 |title=Dynamical approaches to cognitive science |journal=Trends in Cognitive Sciences |volume=4 |issue=3 |pages=91–99 |doi=10.1016/s1364-6613(99)01440-0 |pmid=10689343 |s2cid=16515284 |issn=1364-6613}}</ref> some authors{{which|date=February 2016}} now argue that any split between connectionism and computationalism is more conclusively characterized as a split between computationalism and [[Cognitive Model#Dynamical systems|dynamical systems]].

In 2014, [[Alex Graves (computer scientist)|Alex Graves]] and others from [[DeepMind]] published a series of papers describing a novel Deep Neural Network structure called the [[Neural Turing Machine]]<ref>{{cite arXiv|last1=Graves|first1=Alex|title=Neural Turing Machines|eprint=1410.5401|class=cs.NE|year=2014}}</ref> able to read symbols on a tape and store symbols in memory. Relational Networks, another Deep Network module published by DeepMind, are able to create object-like representations and manipulate them to answer complex questions. Relational Networks and Neural Turing Machines are further evidence that connectionism and computationalism need not be at odds.