Editing Natural language processing (section)

== General tendencies and (possible) future directions ==
Based on long-standing trends in the field, it is possible to extrapolate future directions of NLP. As of 2020, three trends among the topics of the long-standing series of CoNLL Shared Tasks can be observed:<ref>{{Cite web|title=Previous shared tasks {{!}} CoNLL|url=https://www.conll.org/previous-tasks|access-date=2021-01-11|website=www.conll.org}}</ref>

* Interest on increasingly abstract, "cognitive" aspects of natural language (1999–2001: shallow parsing, 2002–03: named entity recognition, 2006–09/2017–18: dependency syntax, 2004–05/2008–09 semantic role labelling, 2011–12 coreference, 2015–16: discourse parsing, 2019: semantic parsing).
* Increasing interest in multilinguality, and, potentially, multimodality (English since 1999; Spanish, Dutch since 2002; German since 2003; Bulgarian, Danish, Japanese, Portuguese, Slovenian, Swedish, Turkish since 2006; Basque, Catalan, Chinese, Greek, Hungarian, Italian, Turkish since 2007; Czech since 2009; Arabic since 2012; 2017: 40+ languages; 2018: 60+/100+ languages)
* Elimination of symbolic representations (rule-based over supervised towards weakly supervised methods, representation learning and end-to-end systems)

=== Cognition ===
Most higher-level NLP applications involve aspects that emulate intelligent behaviour and apparent comprehension of natural language. More broadly speaking, the technical operationalization of increasingly advanced aspects of cognitive behaviour represents one of the developmental trajectories of NLP (see trends among CoNLL shared tasks above).

[[Cognition]] refers to "the mental action or process of acquiring knowledge and understanding through thought, experience, and the senses."<ref>{{Cite web|title=Cognition|url=https://www.lexico.com/definition/cognition|archive-url=https://web.archive.org/web/20200715113427/https://www.lexico.com/definition/cognition|url-status=dead|archive-date=July 15, 2020|website=Lexico|publisher=[[Oxford University Press]] and [[Dictionary.com]]|access-date=6 May 2020}}</ref> [[Cognitive science]] is the interdisciplinary, scientific study of the mind and its processes.<ref>{{cite web|quote=Cognitive science is an interdisciplinary field of researchers from Linguistics, psychology, neuroscience, philosophy, computer science, and anthropology that seek to understand the mind. |url = http://www.aft.org/newspubs/periodicals/ae/summer2002/willingham.cfm |title= Ask the Cognitive Scientist|website = American Federation of Teachers|date = 8 August 2014 }}</ref> [[Cognitive linguistics]] is an interdisciplinary branch of linguistics, combining knowledge and research from both psychology and linguistics.<ref>{{Cite book|title=Handbook of Cognitive Linguistics and Second Language Acquisition|last=Robinson|first=Peter|publisher=Routledge|year=2008|isbn=978-0-805-85352-0|pages=3–8}}</ref> Especially during the age of [[#Symbolic NLP (1950s – early 1990s)|symbolic NLP]], the area of computational linguistics maintained strong ties with cognitive studies.

As an example, [[George Lakoff]] offers a methodology to build natural language processing (NLP) algorithms through the perspective of cognitive science, along with the findings of cognitive linguistics,<ref>{{Cite book|title=Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Philosophy; Appendix: The Neural Theory of Language Paradigm |last= Lakoff |first= George |publisher= New York Basic Books|year=1999|isbn=978-0-465-05674-3|pages=569–583}}</ref> with two defining aspects:

# Apply the theory of [[conceptual metaphor]], explained by Lakoff as "the understanding of one idea, in terms of another" which provides an idea of the intent of the author.<ref>{{Cite book|title=A Cognitive Theory of Cultural Meaning|last= Strauss |first= Claudia |publisher= Cambridge University Press|year=1999|isbn=978-0-521-59541-4|pages=156–164}}</ref> For example, consider the English word ''big''. When used in a comparison ("That is a big tree"), the author's intent is to imply that the tree is ''physically large'' relative to other trees or the authors experience.  When used metaphorically ("Tomorrow is a big day"), the author's intent to imply ''importance''.  The intent behind other usages, like in "She is a big person", will remain somewhat ambiguous to a person and a cognitive NLP algorithm alike without additional information.
# Assign relative measures of meaning to a word, phrase, sentence or piece of text based on the information presented before and after the piece of text being analyzed, e.g., by means of a [[probabilistic context-free grammar]] (PCFG). The mathematical equation for such algorithms is presented in  [https://worldwide.espacenet.com/patent/search/family/055314712/publication/US9269353B1?q=pn%3DUS9269353 US Patent 9269353]:<ref>{{cite patent |country=US |number=9269353|status=patent}}</ref>
::<math> {RMM(token_N)}
=
{PMM(token_N)} 
\times  
\frac{1}{2d}
\left (\sum_{i=-d}^d {((PMM(token_{N})} \times {PF(token_{N-i},token_N,token_{N+i}))_i}\right )
</math>

::''Where''
:::'''RMM''' is the relative measure of meaning
:::'''token''' is any block of text, sentence, phrase or word
:::'''N''' is the number of tokens being analyzed
:::'''PMM''' is the probable measure of meaning based on a corpora
:::'''d''' is the non zero location of the token along the sequence of '''N''' tokens
:::'''PF''' is the probability function specific to a language

Ties with cognitive linguistics are part of the historical heritage of NLP, but they have been less frequently addressed since the statistical turn during the 1990s. Nevertheless, approaches to develop cognitive models towards technically operationalizable frameworks have been pursued in the context of various frameworks, e.g., of cognitive grammar,<ref>{{Cite web|title=Universal Conceptual Cognitive Annotation (UCCA)|url=https://universalconceptualcognitiveannotation.github.io/|access-date=2021-01-11|website=Universal Conceptual Cognitive Annotation (UCCA)|language=en-US}}</ref> functional grammar,<ref>Rodríguez, F. C., & Mairal-Usón, R. (2016). [https://www.redalyc.org/pdf/1345/134549291020.pdf Building an RRG computational grammar]. ''Onomazein'', (34), 86–117.</ref> construction grammar,<ref>{{Cite web|title=Fluid Construction Grammar – A fully operational processing system for construction grammars|url=https://www.fcg-net.org/|access-date=2021-01-11|language=en-US}}</ref> computational psycholinguistics and cognitive neuroscience (e.g., [[ACT-R]]), however, with limited uptake in mainstream NLP (as measured by presence on major conferences<ref>{{Cite web|title=ACL Member Portal {{!}} The Association for Computational Linguistics Member Portal|url=https://www.aclweb.org/portal/|access-date=2021-01-11|website=www.aclweb.org}}</ref> of the [[Association for Computational Linguistics|ACL]]). More recently, ideas of cognitive NLP have been revived as an approach to achieve [[Explainable artificial intelligence|explainability]], e.g., under the notion of "cognitive AI".<ref>{{Cite web|title=Chunks and Rules|url=https://www.w3.org/Data/demos/chunks/chunks.html|access-date=2021-01-11|website=W3C |language=en}}</ref> Likewise, ideas of cognitive NLP are inherent to neural models [[multimodal interaction|multimodal]] NLP (although rarely made explicit)<ref>{{Cite journal|doi=10.1162/tacl_a_00177|title=Grounded Compositional Semantics for Finding and Describing Images with Sentences|year=2014|last1=Socher|first1=Richard|last2=Karpathy|first2=Andrej|last3=Le|first3=Quoc V.|last4=Manning|first4=Christopher D.|last5=Ng|first5=Andrew Y.|journal=Transactions of the Association for Computational Linguistics|volume=2|pages=207–218|s2cid=2317858|doi-access=free}}</ref> and developments in [[artificial intelligence]], specifically tools and technologies using [[large language model]] approaches<ref>{{Cite arXiv|title=Language models show human-like content effects on reasoning, Dasgupta, Lampinen et al|eprint=2207.07051 |language=en |last1=Dasgupta |first1=Ishita |last2=Lampinen |first2=Andrew K. |last3=Chan |first3=Stephanie C. Y. |last4=Creswell |first4=Antonia |last5=Kumaran |first5=Dharshan |last6=McClelland |first6=James L. |last7=Hill |first7=Felix |year=2022 |class=cs.CL }}</ref> and new directions in [[artificial general intelligence]] based on the [[free energy principle]]<ref>{{Cite book|title=Active Inference: The Free Energy Principle in Mind, Brain, and Behavior; Chapter 4 The Generative Models of Active Inference |last= Friston |first= Karl J. |publisher= The MIT Press|year=2022|isbn=978-0-262-36997-8}}</ref> by British neuroscientist and theoretician at University College London [[Karl J. Friston]].