Editing Natural language understanding

{{Short description|Subtopic of natural language processing in artificial intelligence}}
{{hatnote|This article is about the computer processing ability. For the psychological concept, see [[Language processing in the brain]]}}
{{Update|date=February 2024|reason=lack of discussion of recent developments related to [[large language models]], but also no mention of older techniques like [[word embedding]] or [[word2vec]]}}

'''Natural language understanding''' ('''NLU''') or '''natural language interpretation''' ('''NLI''')<ref>Semaan, P. (2012). [http://www.lacsc.org/papers/PaperA6.pdf Natural Language Generation: An Overview]. Journal of Computer Science & Research (JCSCR)-ISSN, 50-57</ref> is a subset of [[natural language processing]] in [[artificial intelligence]] that deals with machine [[reading comprehension]]. NLU has been considered an [[AI-hard]] problem.<ref>Roman V. Yampolskiy. Turing Test as a Defining Feature of AI-Completeness . In Artificial Intelligence, Evolutionary Computation and Metaheuristics (AIECM) --In the footsteps of Alan Turing. Xin-She Yang (Ed.). pp. 3-17. (Chapter 1). Springer, London. 2013. http://cecs.louisville.edu/ry/TuringTestasaDefiningFeature04270003.pdf</ref>

There is considerable commercial interest in the field because of its application to [[automated reasoning]],<ref>Van Harmelen, Frank, Vladimir Lifschitz, and Bruce Porter, eds. [https://books.google.com/books?id=xwBDylHhJhYC&q=%22natural+language+understanding%22 Handbook of knowledge representation]. Vol. 1. Elsevier, 2008.</ref> [[machine translation]],<ref>Macherey, Klaus, Franz Josef Och, and Hermann Ney. "[https://www.researchgate.net/profile/Klaus_Macherey/publication/2371092_Natural_Language_Understanding_Using_Statistical_Machine_Translation/links/00463523b62bc9b5e6000000/Natural-Language-Understanding-Using-Statistical-Machine-Translation.pdf Natural language understanding using statistical machine translation]." Seventh European Conference on Speech Communication and Technology. 2001.</ref> [[question answering]],<ref>Hirschman, Lynette, and Robert Gaizauskas. "[https://www.researchgate.net/profile/Rob_Gaizauskas/publication/231807195_Natural_Language_Question_Answering_The_View_from_Here/links/0c96052a09fa7b819e000000/Natural-Language-Question-Answering-The-View-from-Here.pdf Natural language question answering: the view from here]." natural language engineering 7.4 (2001): 275-300.</ref> news-gathering, [[text categorization]], [[Voice user interface|voice-activation]], archiving, and large-scale [[content analysis]].

==History==
The program [[STUDENT (computer program)|STUDENT]], written in 1964 by [[Daniel Bobrow]] for his PhD dissertation at [[MIT]], is one of the earliest known attempts at NLU by a computer.<ref>[[American Association for Artificial Intelligence]] ''Brief History of AI'' [http://www.aaai.org/AITopics/pmwiki/pmwiki.php/AITopics/BriefHistory]</ref><ref>[[Daniel Bobrow]]'s PhD Thesis [http://hdl.handle.net/1721.1/5922 Natural Language Input for a Computer Problem Solving System].</ref><ref>''Machines who think'' by Pamela McCorduck 2004 {{ISBN|1-56881-205-1}} page 286</ref><ref>Russell, Stuart J.; Norvig, Peter (2003), ''Artificial Intelligence: A Modern Approach'' Prentice Hall, {{ISBN|0-13-790395-2}}, http://aima.cs.berkeley.edu/, p. 19</ref><ref>''Computer Science Logo Style: Beyond programming'' by Brian Harvey 1997 {{ISBN|0-262-58150-7}} page 278</ref> Eight years after [[John McCarthy (computer scientist)|John McCarthy]] coined the term [[artificial intelligence]], Bobrow's dissertation (titled ''Natural Language Input for a Computer Problem Solving System'') showed how a computer could understand simple natural language input to solve algebra word problems.

A year later, in 1965, [[Joseph Weizenbaum]] at MIT wrote [[ELIZA]], an interactive program that carried on a dialogue in English on any topic, the most popular being psychotherapy. ELIZA worked by simple parsing and substitution of key words into canned phrases and Weizenbaum sidestepped the problem of giving the program a [[database]] of real-world knowledge or a rich [[lexicon]]. Yet ELIZA gained surprising popularity as a toy project and can be seen as a very early precursor to current commercial systems such as those used by [[Ask.com]].<ref>Weizenbaum, Joseph (1976). ''Computer power and human reason: from judgment to calculation'' W. H. Freeman and Company. {{ISBN|0-7167-0463-3}} pages 188-189</ref>

In 1969, [[Roger Schank]] at [[Stanford University]] introduced the [[conceptual dependency theory]] for NLU.<ref>[[Roger Schank]], 1969, ''A conceptual dependency parser for natural language'' Proceedings of the 1969 conference on Computational linguistics, Sång-Säby, Sweden, pages 1-3</ref> This model, partially influenced by the work of [[Sydney Lamb]], was extensively used by Schank's students at [[Yale University]], such as [[Robert Wilensky]], [[Wendy Lehnert]], and [[Janet Kolodner]].

In 1970, [[William Aaron Woods|William A. Woods]] introduced the [[augmented transition network]] (ATN) to represent natural language input.<ref>Woods, William A (1970). "Transition Network Grammars for Natural Language Analysis". Communications of the ACM 13 (10): 591–606 [http://www.eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED037733&ERICExtSearch_SearchType_0=no&accno=ED037733]</ref> Instead of ''[[phrase structure rules]]'' ATNs used an equivalent set of [[finite-state automata]] that were called recursively. ATNs and their more general format called "generalized ATNs" continued to be used for a number of years.

In 1971, [[Terry Winograd]] finished writing [[SHRDLU]] for his PhD thesis at MIT. SHRDLU could understand simple English sentences in a restricted world of children's blocks to direct a robotic arm to move items. The successful demonstration of SHRDLU provided significant momentum for continued research in the field.<ref>''Artificial intelligence: critical concepts'', Volume 1 by Ronald Chrisley, Sander Begeer 2000 {{ISBN|0-415-19332-X}} page 89</ref><ref>Terry Winograd's SHRDLU page at Stanford [http://hci.stanford.edu/~winograd/shrdlu/ SHRDLU]</ref> Winograd continued to be a major influence in the field with the publication of his book ''Language as a Cognitive Process''.<ref>Winograd, Terry (1983), ''Language as a Cognitive Process'', Addison–Wesley, Reading, MA.</ref> At Stanford, Winograd would later advise [[Larry Page]], who co-founded [[Google]].<!--does this really belong here?  it seems like trivia-->

In the 1970s and 1980s, the natural language processing group at [[SRI International]] continued research and development in the field. A number of commercial efforts based on the research were undertaken, ''e.g.'', in 1982 [[Gary Hendrix]] formed [[Symantec Corporation]] originally as a company for developing a natural language interface for database queries on personal computers. However, with the advent of mouse-driven [[graphical user interface]]s, Symantec changed direction. A number of other commercial efforts were started around the same time, ''e.g.'', Larry R. Harris at the Artificial Intelligence Corporation and Roger Schank and his students at Cognitive Systems Corp.<ref>Larry R. Harris, ''Research at the Artificial Intelligence corp.'' ACM SIGART Bulletin, issue 79, January 1982 [http://portal.acm.org/citation.cfm?id=1056663.1056670]</ref><ref>''Inside case-based reasoning'' by Christopher K. Riesbeck, Roger C. Schank 1989 {{ISBN|0-89859-767-6}} page xiii</ref> In 1983, Michael Dyer developed the BORIS system at Yale which bore similarities to the work of Roger Schank and W. G. Lehnert.<ref>''In Depth Understanding: A Model of Integrated Process for Narrative Comprehension.''. Michael G. Dyer. MIT Press. {{ISBN|0-262-04073-5}}</ref>

The third millennium saw the introduction of systems using machine learning for text classification, such as the IBM [[Watson (computer)|Watson]]. However, experts debate how much "understanding" such systems demonstrate: ''e.g.'', according to [[John Searle]], Watson did not even understand the questions.<ref>{{Cite news | url=https://www.wsj.com/articles/SB10001424052748703407304576154313126987674 | title=Watson Doesn't Know It Won on 'Jeopardy!'| newspaper=Wall Street Journal| date=23 February 2011| last1=Searle| first1=John}}</ref>

[[John Ball (cognitive scientist)|John Ball]], cognitive scientist and inventor of the [[Patom Theory]], supports this assessment. Natural language processing has made inroads for applications to support human productivity in service and e-commerce, but this has largely been made possible by narrowing the scope of the application. There are thousands of ways to request something in a human language that still defies conventional natural language processing.{{Citation needed|date=February 2024}} According to Wibe Wagemans, "To have a meaningful conversation with machines is only possible when we match every word to the correct meaning based on the meanings of the other words in the sentence – just like a 3-year-old does without guesswork."<ref>{{Cite web |last=Brandon |first=John |date=2016-07-12 |title=What Natural Language Understanding tech means for chatbots |url=https://venturebeat.com/business/what-natural-language-understanding-tech-means-for-chatbots/ |access-date=2024-02-29 |website=VentureBeat |language=en-US}}</ref>

==Scope and context==
The umbrella term "natural language understanding" can be applied to a diverse set of computer applications, ranging from small, relatively simple tasks such as short commands issued to [[robot]]s, to highly complex endeavors such as the full comprehension of newspaper articles or poetry passages. Many real-world applications fall between the two extremes, for instance [[Document classification|text classification]] for the automatic analysis of emails and their routing to a suitable department in a corporation does not require an in-depth understanding of the text,<ref>''An approach to hierarchical email categorization'' by Peifeng Li et al. in ''Natural language processing and information systems'' edited by Zoubida Kedad, Nadira Lammari 2007 {{ISBN|3-540-73350-7}}</ref> but needs to deal with a much larger vocabulary and more diverse syntax than the management of simple queries to database tables with fixed schemata.

Throughout the years various attempts at processing natural language or ''English-like'' sentences presented to computers have taken place at varying degrees of complexity. Some attempts have not resulted in systems with deep understanding, but have helped overall system usability. For example, [[Wayne Ratliff]] originally developed the ''Vulcan'' program with an English-like syntax to mimic the English speaking computer in [[Star Trek]]. Vulcan later became the [[dBase]] system whose easy-to-use syntax effectively launched the personal computer database industry.<ref>[[InfoWorld]], Nov 13, 1989, page 144</ref><ref>[[InfoWorld]], April 19, 1984, page 71</ref> Systems with an easy to use or English-like syntax are, however, quite distinct from systems that use a rich [[lexicon]] and include an internal [[knowledge representation and reasoning|representation]] (often as [[first order logic]]) of the semantics of natural language sentences.

Hence the breadth and depth of "understanding" aimed at by a system determine both the complexity of the system (and the implied challenges) and the types of applications it can deal with. The "breadth" of a system is measured by the sizes of its vocabulary and grammar. The "depth" is measured by the degree to which its understanding approximates that of a fluent native speaker. At the narrowest and shallowest, ''English-like'' command interpreters require minimal complexity, but have a small range of applications. Narrow but deep systems explore and model mechanisms of understanding,<ref>''Building Working Models of Full Natural-Language Understanding in Limited Pragmatic Domains'' by James Mason 2010 [http://www.yorku.ca/jmason/UnderstandingEnglishInLimitedPragmaticDomains.html]</ref> but they still have limited application. Systems that attempt to understand the contents of a document such as a news release beyond simple keyword matching and to judge its suitability for a user are broader and require significant complexity,<ref>''Mining the Web: discovering knowledge from hypertext data'' by Soumen Chakrabarti 2002 {{ISBN|1-55860-754-4}} page 289</ref> but they are still somewhat shallow. Systems that are both very broad and very deep are beyond the current state of the art.

==Components and architecture==
Regardless of the approach used, most NLU systems share some common components. The system needs a [[lexicon]] of the language and a [[parser]] and [[grammar]] rules to break sentences into an internal representation. The construction of a rich lexicon with a suitable [[Ontology (information science)|ontology]] requires significant effort, ''e.g.'', the [[Wordnet]] lexicon required many person-years of effort.<ref>G. A. Miller, R. Beckwith, C. D. Fellbaum, D. Gross, K. Miller. 1990. ''WordNet: An online lexical database''. Int. J. Lexicograph. 3, 4, pp. 235-244.</ref>

The system also needs theory from ''[[semantics]]'' to guide the comprehension. The interpretation capabilities of a language-understanding system depend on the semantic theory it uses. Competing semantic theories of language have specific trade-offs in their suitability as the basis of computer-automated semantic interpretation.<ref>''Using computers in linguistics: a practical guide'' by John Lawler, Helen Aristar Dry 198 {{ISBN|0-415-16792-2}} page 209</ref> These range from ''[[naive semantics]]'' or ''[[stochastic semantic analysis]]'' to the use of ''[[pragmatics]]'' to derive meaning from context.<ref>Naive semantics for natural language understanding'' by Kathleen Dahlgren 1988 {{ISBN|0-89838-287-4}}</ref><ref>''Stochastically-based semantic analysis'' by Wolfgang Minker, [[Alex Waibel]], Joseph Mariani 1999 {{ISBN|0-7923-8571-3}}</ref><ref>''Pragmatics and natural language understanding'' by Georgia M. Green 1996 {{ISBN|0-8058-2166-X}}</ref> [[Semantic parser]]s convert natural-language texts into formal meaning representations.<ref>Wong, Yuk Wah, and [[Raymond J. Mooney]]. "[http://www.aclweb.org/anthology/N06-1056 Learning for semantic parsing with statistical machine translation]." Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics. Association for Computational Linguistics, 2006.</ref>

Advanced applications of NLU also attempt to incorporate logical [[inference]] within their framework. This is generally achieved by mapping the derived meaning into a set of assertions in [[predicate logic]], then using [[logical deduction]] to arrive at conclusions. Therefore, systems based on functional languages such as [[Lisp (programming language)|Lisp]] need to include a subsystem to represent logical assertions, while logic-oriented systems such as those using the language [[Prolog]] generally rely on an extension of the built-in logical representation framework.<ref>''Natural Language Processing Prolog Programmers'' by M. Covington, 1994 {{ISBN|0-13-629478-2}}</ref><ref>''Natural language processing in Prolog'' by Gerald Gazdar, Christopher S. Mellish 1989 {{ISBN|0-201-18053-7}}</ref>

The management of [[context (language use)|context]] in NLU can present special challenges. A large variety of examples and counter examples have resulted in multiple approaches to the [[Formal semantics (natural language)|formal modeling]] of context, each with specific strengths and weaknesses.<ref>''Understanding language understanding'' by Ashwin Ram, Kenneth Moorman 1999 {{ISBN|0-262-18192-4}} page 111</ref><ref>''Formal aspects of context'' by Pierre Bonzon et al 2000 {{ISBN|0-7923-6350-7}}</ref>

==See also==
*[[Computational semantics]]
*[[Computational linguistics]]
*[[Discourse representation theory]]
*[[Deep linguistic processing]]
*[[History of natural language processing]]
*[[Information extraction]]
*[[Mathematica]]<!-- Bot generated title --><ref>[http://blog.wolfram.com/2010/11/16/programming-with-natural-language-is-actually-going-to-work/ Programming with Natural Language Is Actually Going to Work—Wolfram Blog<!-- Bot generated title -->]</ref><ref>{{cite web|last1=Van Valin, Jr|first1=Robert D.|title=From NLP to NLU|url=http://www.isi.hhu.de/fileadmin/redaktion/Oeffentliche_Medien/Fakultaeten/Philosophische_Fakultaet/Sprache_und_Information/Van_Valin_From_NLP_to_NLU.pdf}}</ref><ref>{{cite web|last1=Ball|first1=John|title=multi-lingual NLU by Pat Inc|url=http://pat.ai/|website=Pat.ai}}</ref>
*[[Natural language processing|Natural-language processing]]
*[[Natural-language programming]]
*[[Natural-language user interface]]
**[[Siri (software)]]
**[[Wolfram Alpha]]
*[[Open information extraction]]
*[[Part-of-speech tagging]]
*[[Speech recognition]]

==Notes==
{{Reflist|30em}}

{{Natural Language Processing}}
{{DEFAULTSORT:Natural Language Understanding}}
[[Category:Natural language processing]]