Editing Question answering (section)

==Question answering methods==
Question answering is dependent on a good search [[text corpus|corpus]]; without documents containing the answer, there is little any question answering system can do. Larger collections generally mean better question answering performance, unless the question domain is orthogonal to the collection. [[Data redundancy]] in massive collections, such as the web, means that nuggets of information are likely to be phrased in many different ways in differing contexts and documents,<ref>Lin, J. (2002). [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.457.6371&rep=rep1&type=pdf The Web as a Resource for Question Answering: Perspectives and Challenges]. In ''Proceedings of the Third International Conference on Language Resources and Evaluation'' (LREC 2002).</ref> leading to two benefits:

# If the right information appears in many forms, the question answering system needs to perform fewer complex NLP techniques to understand the text.
# Correct answers can be filtered from [[false positive]]s because the system can rely on versions of the correct answer appearing more times in the corpus than incorrect ones.

Some question answering systems rely heavily on [[automated reasoning]].<ref>Moldovan, Dan, et al. "[http://anthology.aclweb.org/N/N03/N03-1022.pdf Cogex: A logic prover for question answering]." Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1. Association for Computational Linguistics, 2003.</ref><ref>Furbach, Ulrich, Ingo Glöckner, and Björn Pelzer. "[https://web.archive.org/web/20170831043710/https://pdfs.semanticscholar.org/5d8c/a70a0448998162f91e3ce84fac92e10d4ccf.pdf An application of automated reasoning in natural language question answering]." Ai Communications 23.2-3 (2010): 241–265.</ref>

===Open domain question answering===
{{more citations needed section|date=January 2016}}
In [[information retrieval]], an open-domain question answering system tries to return an answer in response to the user's question. The returned answer is in the form of short texts rather than a list of relevant documents.<ref>{{cite book |last1=Sun |first1=Haitian |last2=Dhingra |first2=Bhuwan |last3=Zaheer |first3=Manzil |last4=Mazaitis |first4=Kathryn |last5=Salakhutdinov |first5=Ruslan |last6=Cohen |first6=William |title=  Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing|chapter=Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text |date=2018 |pages=4231–4242 |chapter-url=https://www.aclweb.org/anthology/D18-1455 |location=Brussels, Belgium|doi=10.18653/v1/D18-1455 |arxiv=1809.00782 |s2cid=52154304 }}</ref> The system finds answers by using a combination of techniques from [[computational linguistics]], [[information retrieval]], and [[knowledge representation]].

The system takes a [[natural language]] question as an input rather than a set of keywords, for example: "When is the national day of China?" It then transforms this input sentence into a query in its [[logical form]]. Accepting natural language questions makes the system more user-friendly, but harder to implement, as there are a variety of question types and the system will have to identify the correct one in order to give a sensible answer. Assigning a question type to the question is a crucial task; the entire answer extraction process relies on finding the correct question type and hence the correct answer type.

Keyword [[Data extraction|extraction]] is the first step in identifying the input question type.<ref>{{cite book |last1=Harabagiu |first1=Sanda |last2=Hickl |first2=Andrew |chapter=Methods for using textual entailment in open-domain question answering  |title= Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL - ACL '06 |date=2006 |pages=905–912 |doi=10.3115/1220175.1220289 |chapter-url=https://www.aclweb.org/anthology/P06-1114 |doi-access=free }}</ref> In some cases, words clearly indicate the question type, e.g., "Who", "Where", "When", or "How many"—these words might suggest to the system that the answers should be of type "Person", "Location", "Date", or "Number", respectively. [[part-of-speech tagging|POS (part-of-speech) tagging]] and syntactic parsing techniques can also determine the answer type. In the example above, the subject is "Chinese National Day", the predicate is "is" and the adverbial modifier is "when", therefore the answer type is "Date". Unfortunately, some interrogative words like "Which", "What", or "How" do not correspond to unambiguous answer types: Each can represent more than one type. In situations like this, other words in the question need to be considered. A lexical dictionary such as [[WordNet]] can be used for understanding the context.

Once the system identifies the question type, it uses an [[information retrieval]] system to find a set of documents that contain the correct keywords. A [[Part-of-speech tagging|tagger]] and [[Phrase chunking|NP/Verb Group chunker]] can verify whether the correct entities and relations are mentioned in the found documents. For questions such as "Who" or "Where", a [[named-entity recognition|named-entity recogniser]] finds relevant "Person" and "Location" names from the retrieved documents. {{clarify|reason=who selects? who ranks? what does "ranking" mean in this context?|text=Only the relevant paragraphs are selected for ranking.|date=April 2023}}

A [[vector space model]] can classify the candidate answers. Check{{Who|date=April 2023}} if the answer is of the correct type as determined in the question type analysis stage. An inference technique can validate the candidate answers. A score is then given to each of these candidates according to the number of question words it contains and how close these words are to the candidate—the more and the closer the better. The answer is then translated by parsing into a compact and meaningful representation. In the previous example, the expected output answer is "1st Oct."

===Mathematical question answering===

An open-source, math-aware, question answering system called [[MathQA]], based on [[Ask Platypus]] and [[Wikidata]], was published in 2018.<ref name="SchubotzScharpf2018">{{cite journal|author1=Moritz Schubotz|author2=Philipp Scharpf|display-authors=et al.|title=Introducing MathQA: a Math-Aware question answering system|doi=10.1108/IDD-06-2018-0022| arxiv=1907.01642|date=12 September 2018|journal=Information Discovery and Delivery|volume=46|issue=4|pages=214–224|publisher=Emerald Publishing Limited|doi-access=free}}</ref> MathQA takes an English or Hindi natural language question as input and returns a mathematical formula retrieved from Wikidata as a succinct answer, translated into a computable form that allows the user to insert values for the variables. The system retrieves names and values of variables and common constants from Wikidata if those are available. It is claimed that the system outperforms a commercial computational mathematical knowledge engine on a test set.<ref name="SchubotzScharpf2018">{{cite journal|author1=Moritz Schubotz|author2=Philipp Scharpf|display-authors=et al.|title=Introducing MathQA: a Math-Aware question answering system|doi=10.1108/IDD-06-2018-0022| arxiv=1907.01642|date=12 September 2018|journal=Information Discovery and Delivery|volume=46|issue=4|pages=214–224|publisher=Emerald Publishing Limited|doi-access=free}}</ref> MathQA is hosted by Wikimedia at https://mathqa.wmflabs.org/. In 2022, it was extended to answer 15 math question types.<ref>Scharpf, P. Schubotz, M. Gipp, B. [https://www.gipp.com/wp-content/papercite-data/pdf/scharpf2022.pdf Mining Mathematical Documents for Question Answering via Unsupervised Formula Labeling] ACM/IEEE Joint Conference on Digital Libraries, 2022.</ref>

MathQA methods need to combine natural and formula language. One possible approach is to perform supervised annotation via [[Entity Linking]]. The "ARQMath Task" at [[CLEF]] 2020<ref name=":1">{{Citation|last1=Zanibbi|first1=Richard|date=2020|url=http://dx.doi.org/10.1007/978-3-030-58219-7_15|pages=169–193|place=Cham|publisher=Springer International Publishing|isbn=978-3-030-58218-0|access-date=2021-06-09|last2=Oard|first2=Douglas W.|last3=Agarwal|first3=Anurag|last4=Mansouri|first4=Behrooz|title=Experimental IR Meets Multilinguality, Multimodality, and Interaction |chapter=Overview of ARQMath 2020: CLEF Lab on Answer Retrieval for Questions on Math |series=Lecture Notes in Computer Science |volume=12260 |doi=10.1007/978-3-030-58219-7_15|s2cid=221351064 |url-access=subscription}}</ref> was launched to address the problem of linking newly posted questions from the platform Math [[Stack Exchange]] to existing ones that were already answered by the community. Providing hyperlinks to already answered, semantically related questions helps users to get answers earlier but is a challenging problem because semantic relatedness is not trivial.<ref name=":0">{{Cite book |last=Scharpf |display-authors=etal |url=http://worldcat.org/oclc/1228449497 |title=ARQMath Lab: An Incubator for Semantic Formula Search in zbMATH Open? |date=2020-12-04 |oclc=1228449497}}</ref> The lab was motivated by the fact that 20% of mathematical queries in general-purpose search engines are expressed as well-formed questions.<ref>{{Cite book|last1=Mansouri|first1=Behrooz|last2=Zanibbi|first2=Richard|last3=Oard|first3=Douglas W.|title=2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL) |chapter=Characterizing Searches for Mathematical Concepts |date=June 2019|chapter-url=http://dx.doi.org/10.1109/jcdl.2019.00019|pages=57–66|publisher=IEEE|doi=10.1109/jcdl.2019.00019|isbn=978-1-7281-1547-4|s2cid=198972305}}</ref> The challenge contained two separate sub-tasks. Task 1: "Answer retrieval" matching old post answers to newly posed questions, and Task 2: "Formula retrieval" matching old post formulae to new questions. Starting with the domain of mathematics, which involves formula language, the goal is to later extend the task to other domains (e.g., STEM disciplines, such as chemistry, biology, etc.), which employ other types of special notation (e.g., chemical formulae).<ref name=":1" /><ref name=":0" />

The inverse of mathematical question answering—mathematical question generation—has also been researched. The PhysWikiQuiz physics question generation and test engine retrieves mathematical formulae from Wikidata together with semantic information about their constituting identifiers (names and values of variables).<ref>{{Cite journal |last1=Scharpf |first1=Philipp |last2=Schubotz |first2=Moritz |last3=Spitz |first3=Andreas |last4=Greiner-Petter |first4=Andre |last5=Gipp |first5=Bela |date=2022 |title=Collaborative and AI-aided Exam Question Generation using Wikidata in Education |doi=10.13140/RG.2.2.30988.18568 |arxiv=2211.08361|s2cid=253270181 }}</ref> The formulae are then rearranged to generate a set of formula variants. Subsequently, the variables are substituted with random values to generate a large number of different questions suitable for individual student tests. PhysWikiquiz is hosted by Wikimedia at https://physwikiquiz.wmflabs.org/.