Editing Natural language generation (section)

==Applications==
===Automatic report generation===

From a commercial perspective, the most successful NLG applications
have been ''data-to-text'' systems which [[automatic summarization|generate textual summaries]] of databases and data sets; these
systems usually perform [[data analysis]] as well as text generation. Research has shown that textual summaries can be more effective than graphs and other visuals for decision support,<ref>{{cite journal |vauthors=Law A, Freer Y, Hunter J, Logie R, McIntosh N, Quinn J |title=A Comparison of Graphical and Textual Presentations of Time Series Data to Support Medical Decision Making in the Neonatal Intensive Care Unit |journal=Journal of Clinical Monitoring and Computing |volume=19 |pages=183–94 |year=2005 |doi=10.1007/s10877-005-0879-3 |pmid=16244840 |issue=3|s2cid=5569544 }}</ref><ref>{{cite journal |vauthors=Gkatzia D, Lemon O, Reiser V|title=Data-to-Text Generation Improves Decision-Making Under Uncertainty|journal=IEEE Computational Intelligence Magazine|volume=12|issue=3|pages=10–17|year=2017|doi=10.1109/MCI.2017.2708998|s2cid=9544295|url=https://napier-surface.worktribe.com/687579/1/gui-journal-paper-27Apr.pdf}}</ref><ref>{{Cite web | url=https://ehudreiter.com/2016/12/26/text-or-graphics/ | title=Text or Graphics?| date=2016-12-26}}</ref> and that computer-generated texts can be superior (from the reader's perspective) to human-written texts.<ref>{{cite journal |vauthors=Reiter E, Sripada S, Hunter J, Yu J, Davy I |title=Choosing Words in Computer-Generated Weather Forecasts |journal=Artificial Intelligence |volume=167 |issue= 1–2|pages=137–69 |year=2005 |doi=10.1016/j.artint.2005.06.006|doi-access=free }}</ref>

The first commercial data-to-text systems produced weather forecasts from weather data. The earliest such system to be deployed was FoG,<ref name=fog>{{cite journal |vauthors=Goldberg E, Driedger N, Kittredge R |title=Using Natural-Language Processing to Produce Weather Forecasts |journal=IEEE Expert |volume=9 |pages=45–53 |year=1994 |doi= 10.1109/64.294135 |issue= 2|s2cid=9709337 }}</ref> which was used by Environment Canada to generate weather forecasts in French and English in the early 1990s.  The success of FoG triggered other work, both research and commercial. Recent applications include the [[Met Office|UK Met Office's]] text-enhanced forecast.<ref>S Sripada, N Burnett, R Turner, J Mastin, D Evans(2014). [http://www.aclweb.org/anthology/W/W14/W14-4401.pdf Generating A Case Study: NLG meeting Weather Industry Demand for Quality and Quantity of Textual Weather Forecasts.] ''Proceedings of INLG 2014''</ref>

Data-to-text systems have since been applied in a range of settings. Following the minor earthquake near Beverly Hills, California on March 17, 2014, The Los Angeles Times reported details about the time, location and strength of the quake within 3 minutes of the event. This report was automatically generated by a 'robo-journalist', which converted the incoming data into text via a preset template.<ref>{{Cite web |last1=Schwencke |first1=Ken Schwencke Ken |last2=Journalist |first2=A. |last3=Programmer |first3=Computer |last4=in 2014 |first4=left the Los Angeles Times |date=2014-03-17 |title=Earthquake aftershock: 2.7 quake strikes near Westwood |url=https://www.latimes.com/local/lanow/earthquake-27-quake-strikes-near-westwood-california-rdivor-story.html |access-date=2022-06-03 |website=Los Angeles Times |language=en-US}}</ref><ref>{{Cite web |last=Levenson |first=Eric |date=2014-03-17 |title=L.A. Times Journalist Explains How a Bot Wrote His Earthquake Story for Him |url=https://www.theatlantic.com/technology/archive/2014/03/earthquake-bot-los-angeles-times/359261/ |access-date=2022-06-03 |website=The Atlantic |language=en}}</ref> Currently there is considerable commercial interest in using NLG to summarise financial and business data.  Indeed, [[Gartner]] has said that NLG will become a standard feature of 90% of modern BI and analytics platforms.<ref>{{Cite web | url=https://www.gartner.com/smarterwithgartner/nueral-networks-and-modern-bi-platforms-will-evolve-data-and-analytics/ | title=Neural Networks and Modern BI Platforms Will Evolve Data and Analytics}}</ref>  NLG is also being used commercially in [[automated journalism]], [[chatbot]]s, generating product descriptions for e-commerce sites, summarising medical records,<ref>{{cite conference |author=Harris MD |title=Building a Large-Scale Commercial NLG System for an EMR  |book-title=Proceedings of the Fifth International Natural Language Generation Conference  |pages=157–60  |year=2008 |url=http://www.aclweb.org/anthology/W08-1120.pdf }}</ref><ref name="portet">{{cite journal |vauthors=Portet F, Reiter E, Gatt A, Hunter J, Sripada S, Freer Y, Sykes C |title=Automatic Generation of Textual Summaries from Neonatal Intensive Care Data |journal=Artificial Intelligence |volume=173 |pages=789–816 |year=2009 |doi=10.1016/j.artint.2008.12.002 |issue=7–8|url=https://hal.archives-ouvertes.fr/hal-00953707/file/aij-bt45.pdf }}</ref> and enhancing [[accessibility]] (for example by describing graphs and data sets to blind people<ref>{{cite web |url=http://www.inf.udec.cl/~leo/iGraph.html |title=Welcome to the iGraph-Lite page |website=www.inf.udec.cl |url-status=dead |archive-url=https://web.archive.org/web/20100316130751/http://www.inf.udec.cl/~leo/igraph.html |archive-date=2010-03-16}}</ref>).

An example of an interactive use of NLG is the [[WYSIWYM (Meant)|WYSIWYM]] framework. It stands for ''What you see is what you meant'' and allows users to see and manipulate the continuously rendered view (NLG output) of an underlying formal language document (NLG input), thereby editing the formal language without learning it.

Looking ahead, the current progress in data-to-text generation paves the way for tailoring texts to specific audiences. For example, data from babies in neonatal care can be converted into text differently in a clinical setting, with different levels of technical detail and explanatory language, depending on intended recipient of the text (doctor, nurse, patient). The same idea can be applied in a sports setting, with different reports generated for fans of specific teams.<ref name=":0">{{cite arXiv |last1=Gatt |first1=Albert |last2=Krahmer |first2=Emiel |date=2018-01-29 |title=Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation |class=cs.CL |eprint=1703.09902 }}</ref>

===Image captioning===

Over the past few years, there has been an increased interest in [[automatic image annotation|automatically generating captions]] for images, as part of a broader endeavor to investigate the interface between vision and language. A case of data-to-text generation, the algorithm of image captioning (or automatic image description) involves taking an image, analyzing its visual content, and generating a textual description (typically a sentence) that verbalizes the most prominent aspects of the image.

An image captioning system involves two sub-tasks. In Image Analysis, features and attributes of an image are detected and labelled, before mapping these outputs to linguistic structures. Recent research utilize''s'' deep learning approaches through features from a pre-trained [[convolutional neural network]] such as AlexNet, VGG or Caffe, where caption generators use an activation layer from the pre-trained network as their input features. Text Generation, the second task, is performed using a wide range of techniques. For example, in the Midge system, input images are represented as triples consisting of object/stuff detections, action/[[pose (computer vision)|pose]] detections and spatial relations. These are subsequently mapped to <noun, verb, preposition> triples and realized using a tree substitution grammar.<ref name=":0" />

A common method in image captioning is to use a vision model (such as a [[Residual neural network|ResNet]]) to encode an image into a vector, then use a language model (such as an [[Recurrent neural network|RNN]]) to decode the vector into a caption.<ref>{{Cite journal |last=Vinyals |first=Oriol |last2=Toshev |first2=Alexander |last3=Bengio |first3=Samy |last4=Erhan |first4=Dumitru |date=2015 |title=Show and Tell: A Neural Image Caption Generator |url=https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Vinyals_Show_and_Tell_2015_CVPR_paper.html |pages=3156–3164}}</ref><ref>{{Cite journal |last=Karpathy |first=Andrej |last2=Fei-Fei |first2=Li |date=2015 |title=Deep Visual-Semantic Alignments for Generating Image Descriptions |url=https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Karpathy_Deep_Visual-Semantic_Alignments_2015_CVPR_paper.html |pages=3128–3137}}</ref>

Despite advancements, challenges and opportunities remain in image capturing research. Notwithstanding the recent introduction of Flickr30K, MS COCO and other large datasets ''have'' enabled the training of more complex models such as neural networks, it has been argued that ''research in image captioning could benefit from larger and diversified datasets.'' Designing automatic measures that can mimic human judgments in evaluating the suitability of image descriptions is another need in the area. Other open challenges include visual [[question answering|question-answering]] (VQA),<ref>{{cite conference |last1=Kodali |first1=Venkat |last2=Berleant |first2=Daniel |title=Recent, Rapid Advancement in Visual Question Answering Architecture: a Review |book-title=Proceedings of the 22nd IEEE International Conference on EIT |pages=133–146 |year=2022 |arxiv=2203.01322 }}</ref> as well as the construction and evaluation multilingual repositories for image description.<ref name=":0" />

===Chatbots===

Another area  where NLG has been widely applied is automated [[dialogue]] systems, frequently in the form of chatbots. A [[chatbot]] or chatterbot is a [[Software agent|software]] application used to conduct an on-line chat [[conversation]] via text or [[Speech synthesis|text-to-speech]], in lieu of providing direct contact with a live human agent. While [[natural language processing]] (NLP) techniques are applied in deciphering human input, NLG informs the output part of the chatbot algorithms in facilitating real-time dialogues.

Early chatbot systems, including [[Cleverbot]] created by Rollo Carpenter in 1988 and published in 1997,{{Citation needed|date=January 2023}}  reply to questions by identifying how a human has responded to the same question in a conversation database using [[information retrieval]] (IR) techniques.{{Citation needed|date=January 2023}} Modern chatbot systems predominantly rely on machine learning (ML) models, such as sequence-to-sequence learning and reinforcement learning to generate natural language output. Hybrid models have also been explored. For example, the Alibaba shopping assistant first uses an IR approach to retrieve the best candidates from the knowledge base, then uses the  ML-driven seq2seq model re-rank the candidate responses and generate the answer.<ref>{{cite arXiv |last=Mnasri |first=Maali |date=2019-03-21 |title=Recent advances in conversational NLP: Towards the standardization of Chatbot building |class=cs.CL |eprint=1903.09025 }}</ref>

===Creative writing and computational humor===

Creative language generation by NLG has been hypothesized since the field's origins. A recent pioneer in the area is Phillip Parker, who has developed an arsenal of algorithms capable of automatically generating textbooks, crossword puzzles, poems and books on topics ranging from bookbinding to cataracts.<ref>{{Cite web |date=2013-02-11 |title=How To Author Over 1 Million Books |url=https://www.huffpost.com/entry/philip-parker-books_n_2648820 |access-date=2022-06-03 |website=HuffPost |language=en}}</ref> The advent of large pretrained transformer-based language models such as GPT-3 has also enabled breakthroughs, with such models demonstrating recognizable ability for creating-writing tasks.<ref>{{Cite web |title=Exploring GPT-3: A New Breakthrough in Language Generation |url=https://www.kdnuggets.com/exploring-gpt-3-a-new-breakthrough-in-language-generation.html/ |access-date=2022-06-03 |website=KDnuggets |language=en-US}}</ref>

A related area of NLG application is computational humor production.  JAPE (Joke Analysis and Production Engine) is one of the earliest large, automated humor production systems that uses a hand-coded template-based approach to create punning riddles for children. HAHAcronym creates humorous reinterpretations of any given acronym, as well as proposing new fitting acronyms given some keywords.<ref name=":1">{{Cite journal |last=Winters |first=Thomas |date=2021-04-30 |title=Computers Learning Humor Is No Joke |journal=Harvard Data Science Review |url=https://hdsr.mitpress.mit.edu/pub/wi9yky5c/release/3 |language=en |volume=3 |issue=2 |doi=10.1162/99608f92.f13a2337|s2cid=235589737 |doi-access=free }}</ref>

Despite progresses, many challenges remain in producing automated creative and humorous content that rival human output. In an experiment for generating satirical headlines, outputs of their best BERT-based model were perceived as funny 9.4% of the time (while real headlines from [[The Onion]] were 38.4%) and a GPT-2 model fine-tuned on satirical headlines achieved 6.9%.<ref>{{Cite journal |last1=Horvitz |first1=Zachary |last2=Do |first2=Nam |last3=Littman |first3=Michael L. |date=July 2020 |title=Context-Driven Satirical News Generation |url=https://aclanthology.org/2020.figlang-1.5 |journal=Proceedings of the Second Workshop on Figurative Language Processing |location=Online |publisher=Association for Computational Linguistics |pages=40–50 |doi=10.18653/v1/2020.figlang-1.5|s2cid=220330989 |doi-access=free }}</ref>  It has been pointed out that two main issues with humor-generation systems are the lack of annotated data sets and the lack of formal evaluation methods,<ref name=":1" /> which could be applicable to other creative content generation. Some have argued relative to other applications, there has been a lack of attention to creative aspects of language production within NLG. NLG researchers stand to benefit from insights into what constitutes creative language production, as well as structural features of narrative that have the potential to improve NLG output even in data-to-text systems.<ref name=":0" />