Editing Speech synthesis

{{short description|Artificial production of human speech}}
{{pp-pc}}
{{listen
| filename     = JärDa-utrop.ogg
| title        = Automatic announcement
| description  = A synthetic voice announcing an arriving train in Sweden.
| format       = [[Ogg]]
}}

'''Speech synthesis''' is the artificial production of human [[speech]]. A computer system used for this purpose is called a '''speech synthesizer''', and can be implemented in [[software]] or [[Computer hardware|hardware]] products. A '''text-to-speech''' ('''TTS''') system converts normal language text into speech; other systems render [[symbolic linguistic representation]]s like [[phonetic transcription]]s into speech.<ref>{{Cite book |first1=Jonathan |last1=Allen |first2=M. Sharon |last2=Hunnicutt |first3=Dennis |last3=Klatt |title=From Text to Speech: The MITalk system |publisher=Cambridge University Press |year=1987 |isbn=978-0-521-30641-6 |url-access=registration |url=https://archive.org/details/fromtexttospeech00alle }}</ref> The reverse process is [[speech recognition]].

Synthesized speech can be created by [[Concatenative synthesis|concatenating]] pieces of recorded speech that are stored in a [[database]]. Systems differ in the size of the stored speech units; a system that stores [[phone (phonetics)|phones]] or [[diphone]]s provides the largest output range, but may lack clarity.{{Citation needed|date=September 2024}} For specific usage domains, the storage of entire words or sentences allows for high-quality output. Alternatively, a synthesizer can incorporate a model of the [[vocal tract]] and other human voice characteristics to create a completely "synthetic" voice output.<ref>{{Cite journal | doi = 10.1121/1.386780 | last1 = Rubin | first1 = P. | last2 = Baer | first2 = T. | last3 = Mermelstein | first3 = P. | year = 1981 | title = An articulatory synthesizer for perceptual research | journal = Journal of the Acoustical Society of America | volume = 70 | issue = 2| pages = 321–328 | bibcode = 1981ASAJ...70..321R }}</ref>

The quality of a speech synthesizer is judged by its similarity to the human voice and by its ability to be understood clearly. An intelligible text-to-speech program allows people with [[visual impairment]]s or [[reading disability|reading disabilities]] to listen to written words on a home computer. Many computer [[operating system]]s have included speech synthesizers since the early 1990s.{{Citation needed|date=September 2024}}

[[File:TTS System.svg|550px|thumb|Overview of a typical TTS system]]

A text-to-speech system (or "engine") is composed of two parts:<ref>{{Cite book |first1=Jan P. H. |last1=van Santen |first2=Richard W. |last2=Sproat |first3=Joseph P. |last3=Olive |first4=Julia |last4=Hirschberg |title=Progress in Speech Synthesis |publisher=Springer |year=1997 |isbn=978-0-387-94701-3 |url-access=registration |url=https://archive.org/details/progressinspeech0000unse }}</ref> a [[Input method|front-end]] and a [[Front and back ends|back-end]]. The front-end has two major tasks. First, it converts raw text containing symbols like numbers and abbreviations into the equivalent of written-out words. This process is often called ''[[text normalization]]'', ''pre-processing'', or ''[[Tokenization (lexical analysis)|tokenization]]''. The front-end then assigns [[phonetic transcription]]s to each word, and divides and marks the text into [[prosody (linguistics)|prosodic units]], like [[phrase]]s, [[clause]]s, and [[sentence (linguistics)|sentence]]s. The process of assigning phonetic transcriptions to words is called ''text-to-phoneme'' or ''[[grapheme]]-to-phoneme'' conversion. Phonetic transcriptions and [[Prosody (linguistics)|prosody]] information together make up the symbolic linguistic representation that is output by the front-end. The back-end—often referred to as the ''synthesizer''—then converts the symbolic linguistic representation into sound. In certain systems, this part includes the computation of the ''target prosody'' (pitch contour, phoneme durations),<ref>{{Cite journal
| last1 = Van Santen | first1 = J.
| title = Assignment of segmental duration in text-to-speech synthesis
| doi = 10.1006/csla.1994.1005
| journal = Computer Speech & Language
| volume = 8
| issue = 2
| pages = 95–128
|date=April 1994
}}</ref> which is then imposed on the output speech.

== History ==

Long before the invention of [[electronics|electronic]] [[signal processing]], some people tried to build machines to emulate human speech.{{citation needed|date=November 2024}} There were also legends of the existence of "[[Brazen Head]]s", such as those involving Pope [[Silvester II]] (d. 1003 AD), [[Albertus Magnus]] (1198–1280), and [[Roger Bacon]] (1214–1294).

In 1779, the [[Germany|German]]-[[Denmark|Danish]] scientist [[Christian Gottlieb Kratzenstein]] won the first prize in a competition announced by the Russian [[Russian Academy of Sciences|Imperial Academy of Sciences and Arts]] for models he built of the human [[vocal tract]] that could produce the five long [[vowel]] sounds (in [[International Phonetic Alphabet]] notation: {{IPA|[aː]}}, {{IPA|[eː]}}, {{IPA|[iː]}}, {{IPA|[oː]}} and {{IPA|[uː]}}).<ref name="Helsinki">[http://www.acoustics.hut.fi/publications/files/theses/lemmetty_mst/chap2.html History and Development of Speech Synthesis], Helsinki University of Technology, Retrieved on November 4, 2006</ref> There followed the [[bellows]]-operated "[[Wolfgang von Kempelen's Speaking Machine|acoustic-mechanical speech machine]]" of [[Wolfgang von Kempelen]] of [[Pressburg]], Hungary, described in a 1791 paper.<ref>''Mechanismus der menschlichen Sprache nebst der Beschreibung seiner sprechenden Maschine'' ("Mechanism of the human speech with description of its speaking machine", J. B. Degen, Wien). {{in lang|de}}</ref> This machine added models of the tongue and lips, enabling it to produce consonants as well as vowels. In 1837, [[Charles Wheatstone]] produced a "speaking machine" based on von Kempelen's design, and in 1846, Joseph Faber exhibited the "[[Euphonia (device)|Euphonia]]". In 1923, Paget resurrected Wheatstone's design.<ref>{{Cite journal|last=Mattingly|first=Ignatius G.|year=1974|editor1-last=Sebeok|editor1-first=Thomas A.|title=Speech synthesis for phonetic and phonological models|url=http://www.haskins.yale.edu/Reprints/HL0173.pdf|url-status=dead|journal=Current Trends in Linguistics|location=Mouton, The Hague|volume=12|pages=2451–2487|archive-url=https://web.archive.org/web/20130512085755/http://www.haskins.yale.edu/Reprints/HL0173.pdf|archive-date=2013-05-12|access-date=2011-12-13}}</ref>

In the 1930s, [[Bell Labs]] developed the [[vocoder]],  which automatically analyzed speech into its fundamental tones and resonances. From his work on the vocoder, [[Homer Dudley]] developed a keyboard-operated voice-synthesizer called [[The Voder]] (Voice Demonstrator), which he exhibited at the [[1939 New York World's Fair]].

[[Franklin S. Cooper|Dr. Franklin S. Cooper]] and his colleagues at [[Haskins Laboratories]] built the [[Pattern playback]] in the late 1940s and completed it in 1950. There were several different versions of this hardware device; only one currently survives. The machine converts pictures of the acoustic patterns of speech in the form of a spectrogram back into sound. Using this device, [[Alvin Liberman]] and colleagues discovered acoustic cues for the perception of [[phonetic]] segments (consonants and vowels).

=== Electronic devices ===
[[File:Computer and speech synthesiser housing, 19 (9663804888).jpg|thumb| Computer and speech synthesizer housing used by [[Stephen Hawking]] in 1999]]
The first computer-based speech-synthesis systems originated in the late 1950s. Noriko Umeda ''et al.'' developed the first general English text-to-speech system in 1968, at the [[Electrotechnical Laboratory]] in Japan.<ref>{{cite journal | last1 = Klatt | first1 = D | year = 1987 | title = Review of text-to-speech conversion for English | journal = Journal of the Acoustical Society of America | volume = 82 | issue = 3| pages = 737–93 | doi= 10.1121/1.395275| pmid = 2958525 | bibcode = 1987ASAJ...82..737K }}</ref> In 1961, physicist [[John Larry Kelly, Jr]] and his colleague [[Louis Gerstman]]<ref>{{cite news|last=Lambert|first=Bruce|date=March 21, 1992|title=Louis Gerstman, 61, a Specialist In Speech Disorders and Processes|work=The New York Times|url=https://www.nytimes.com/1992/03/21/nyregion/louis-gerstman-61-a-specialist-in-speech-disorders-and-processes.html}}</ref> used an [[IBM 704]] computer to synthesize speech, an event among the most prominent in the history of [[Bell Labs]].{{citation needed|date=April 2016}} Kelly's voice recorder synthesizer ([[vocoder]]) recreated the song "[[Daisy Bell]]", with musical accompaniment from [[Max Mathews]]. Coincidentally, [[Arthur C. Clarke]] was visiting his friend and colleague John Pierce at the Bell Labs Murray Hill facility. Clarke was so impressed by the demonstration that he used it in the climactic scene of his screenplay for his novel ''[[2001: A Space Odyssey (novel)|2001: A Space Odyssey]]'',<ref>{{cite web|url=http://www.lsi.usp.br/~rbianchi/clarke/ACC.Biography.html |title=Arthur C. Clarke Biography |access-date=5 December 2017 |url-status=dead |archive-url=https://web.archive.org/web/19971211154551/http://www.lsi.usp.br/~rbianchi/clarke/ACC.Biography.html |archive-date=December 11, 1997 }}</ref> where the [[HAL 9000]] computer sings the same song as astronaut [[David Bowman (Space Odyssey)|Dave Bowman]] puts it to sleep.<ref>{{cite web|url=http://www.bell-labs.com/news/1997/march/5/2.html |title=Where "HAL" First Spoke (Bell Labs Speech Synthesis website) |publisher=Bell Labs |access-date=2010-02-17 |url-status=dead |archive-url=https://web.archive.org/web/20000407081031/http://www.bell-labs.com/news/1997/march/5/2.html |archive-date=2000-04-07 }}</ref> Despite the success of purely electronic speech synthesis, research into mechanical speech-synthesizers continues.<ref>[http://www.takanishi.mech.waseda.ac.jp/top/research/voice/index.htm Anthropomorphic Talking Robot Waseda-Talker Series] {{webarchive|url=https://web.archive.org/web/20160304034116/http://www.takanishi.mech.waseda.ac.jp/top/research/voice/index.htm |date=2016-03-04 }}</ref>{{Third-party inline|date=July 2019}}

[[Linear predictive coding]] (LPC), a form of [[speech coding]], began development with the work of [[Fumitada Itakura]] of [[Nagoya University]] and Shuzo Saito of [[Nippon Telegraph and Telephone]] (NTT) in 1966. Further developments in LPC technology were made by [[Bishnu S. Atal]] and [[Manfred R. Schroeder]] at [[Bell Labs]] during the 1970s.<ref>{{cite journal |last1=Gray |first1=Robert M. |title=A History of Realtime Digital Speech on Packet Networks: Part II of Linear Predictive Coding and the Internet Protocol |journal=Found. Trends Signal Process. |date=2010 |volume=3 |issue=4 |pages=203–303 |doi=10.1561/2000000036 |url=https://ee.stanford.edu/~gray/lpcip.pdf |archive-url=https://ghostarchive.org/archive/20221009/https://ee.stanford.edu/~gray/lpcip.pdf |archive-date=2022-10-09 |url-status=live |issn=1932-8346|doi-access=free }}</ref> LPC was later the basis for early speech synthesizer chips, such as the [[Texas Instruments LPC Speech Chips]] used in the [[Speak & Spell (toy)|Speak & Spell]] toys from 1978.

In 1975, Fumitada Itakura developed the [[line spectral pairs]] (LSP) method for high-compression speech coding, while at NTT.<ref>{{cite journal |last1=Zheng |first1=F. |last2=Song |first2=Z. |last3=Li |first3=L. |last4=Yu |first4=W. |title=The Distance Measure for Line Spectrum Pairs Applied to Speech Recognition |journal=Proceedings of the 5th International Conference on Spoken Language Processing (ICSLP'98) |date=1998 |issue=3 |pages=1123–6 |url=http://www.work.caltech.edu/~ling/pub/icslp98lsp.pdf |archive-url=https://ghostarchive.org/archive/20221009/http://www.work.caltech.edu/~ling/pub/icslp98lsp.pdf |archive-date=2022-10-09 |url-status=live}}</ref><ref name="ieee">{{cite web |title=List of IEEE Milestones |url=https://ethw.org/Milestones:List_of_IEEE_Milestones |publisher=[[IEEE]] |access-date=15 July 2019}}</ref><ref name=ItakuraHistory>{{cite web|url=https://ethw.org/Oral-History:Fumitada_Itakura|title=Fumitada Itakura Oral History|publisher=IEEE Global History Network|date=20 May 2009|access-date=2009-07-21}}</ref> From 1975 to 1981, Itakura studied problems in speech analysis and synthesis based on the LSP method.<ref name=ItakuraHistory/> In 1980, his team developed an LSP-based speech synthesizer chip. LSP is an important technology for speech synthesis and coding, and in the 1990s was adopted by almost all international speech coding standards as an essential component, contributing to the enhancement of digital speech communication over mobile channels and the internet.<ref name="ieee"/>

In 1975, [[MUSA (MUltichannel Speaking Automaton)|MUSA]] was released, and was one of the first Speech Synthesis systems. It consisted of a stand-alone computer hardware and a specialized software that enabled it to read Italian. A second version, released in 1978, was also able to sing Italian in an "[[a cappella]]" style.<ref>{{cite journal |last1=Billi |first1=Roberto |last2=Canavesio |first2=Franco |last3=Ciaramella |first3=Alberto | author-link3 = Alberto Ciaramella|last4=Nebbia |first4=Luciano |title=Interactive voice technology at work: The CSELT experience |journal=Speech Communication |date=1 November 1995 |volume=17 |issue=3 |pages=263–271 |doi=10.1016/0167-6393(95)00030-R}}</ref>
[[File:DECtalk demo.flac|thumb|DECtalk demo recording using the Perfect Paul and Uppity Ursula voices]]
Dominant systems in the 1980s and 1990s were the [[DECtalk]] system, based largely on the work of [[Dennis H. Klatt|Dennis Klatt]] at MIT, and the Bell Labs system;<ref>{{Cite book |first1= Richard W. |last1= Sproat |title= Multilingual Text-to-Speech Synthesis: The Bell Labs Approach |publisher= Springer |year= 1997 |isbn= 978-0-7923-8027-6}}</ref> the latter was one of the first multilingual language-independent systems, making extensive use of [[natural language processing]] methods.

[[File:DNC(Differentiable Neural Computer).png|600px|frameless|center]]
[[File:Fidelity Chess Challenger Voice.jpg|thumb|Fidelity Voice Chess Challenger (1979), the first talking chess computer]]
[[File:Fidelity Chess Challenger Voice speech output.flac|thumb|Speech output from Fidelity Voice Chess Challenger]]

[[Handheld]] electronics featuring speech synthesis began emerging in the 1970s. One of the first was the [[Telesensory Systems|Telesensory Systems Inc.]] (TSI) ''Speech+'' portable calculator for the blind in 1976.<ref>[TSI Speech+ & other speaking calculators]</ref><ref>Gevaryahu, Jonathan, [ "TSI S14001A Speech Synthesizer LSI Integrated Circuit Guide"]{{dead link|date= December 2011}}</ref> Other devices had primarily educational purposes, such as the [[Speak & Spell (toy)|Speak & Spell toy]] produced by [[Texas Instruments]] in 1978.<ref>Breslow, et al. {{patent|US|4326710|title=Talking electronic game}}: "Talking electronic game", April 27, 1982</ref> Fidelity released a speaking version of its electronic chess computer in 1979.<ref>[http://www.ismenio.com/chess_fidelity_vcc.html Voice Chess Challenger]</ref> The first [[video game]] to feature speech synthesis was the 1980 [[shoot 'em up]] [[arcade game]], ''[[Stratovox]]'' (known in Japan as ''Speak & Rescue''), from [[Sunsoft|Sun Electronics]].<ref>[http://www.gamesradar.com/f/gamings-most-important-evolutions/a-20101008102331322035/p-2 Gaming's most important evolutions] {{webarchive|url=https://web.archive.org/web/20110615221800/http://www.gamesradar.com/f/gamings-most-important-evolutions/a-20101008102331322035/p-2 |date=2011-06-15 }}, [[GamesRadar]]</ref><ref>{{cite magazine |last=Adlum |first=Eddie |title=The Replay Years: Reflections from Eddie Adlum |magazine=RePlay |date=November 1985 |volume=11 |issue=2 |pages=134-175 (160-3) |url=https://archive.org/details/re-play-volume-11-issue-no.-2-november-1985-600DPI/RePlay%20-%20Volume%2011%2C%20Issue%20No.%202%20-%20November%201985/page/162/mode/2up}}</ref> The first [[personal computer game]] with speech synthesis was ''[[Stealth game#History|Manbiki Shoujo]]'' (''Shoplifting Girl''), released in 1980 for the [[PET 2001]], for which the game's developer, Hiroshi Suzuki, developed a "''zero cross''" programming technique to produce a synthesized speech waveform.<ref>{{cite book |last=Szczepaniak |first=John |year=2014 |title=The Untold History of Japanese Game Developers |publisher=SMG Szczepaniak |volume=1 |pages=544–615 |isbn=978-0992926007 }}</ref> Another early example, the arcade version of ''[[Berzerk (video game)|Berzerk]]'', also dates from 1980. The [[Milton Bradley Company]] produced the first multi-player [[electronic game]] using voice synthesis, ''[[Milton (game)|Milton]]'', in the same year.

In 1976, Computalker Consultants released their CT-1 Speech Synthesizer. Designed by D. Lloyd Rice and Jim Cooper, it was an analog synthesizer built to work with microcomputers using the S-100 bus standard.<ref>{{Cite news |title=A Short History of Computalker |url=https://amhistory.si.edu/archives/speechsynthesis/ss_rice.htm |website=Smithsonian Speech Synthesis History Project}}</ref>

Early electronic speech-synthesizers sounded robotic and were often barely intelligible. The quality of synthesized speech has steadily improved, but {{as of | 2016 | lc = on}} output from contemporary speech synthesis systems remains clearly distinguishable from actual human speech.

Synthesized voices typically sounded male until 1990, when [[Ann Syrdal]], at [[AT&T Bell Laboratories]], created a female voice.<ref name=NewYorkTimes>{{cite news|url=https://www.nytimes.com/2020/08/20/technology/ann-syrdal-who-helped-give-computers-a-female-voice-dies-at-74.html|title=Ann Syrdal, Who Helped Give Computers a Female Voice, Dies at 74|work=The New York Times|date=2020-08-20|author=CadeMetz|access-date=2020-08-23}}</ref>

Kurzweil predicted in 2005 that as the [[cost-performance ratio]] caused speech synthesizers to become cheaper and more accessible, more people would benefit from the use of text-to-speech programs.<ref>{{cite book
  |last = Kurzweil
  |first = Raymond
  |author-link = Raymond Kurzweil
  |title = The Singularity is Near
  |publisher = [[Penguin Books]]
  |year = 2005
  |isbn = 978-0-14-303788-0}}
</ref>

== Synthesizer technologies ==

The most important qualities of a speech synthesis system are ''naturalness'' and ''[[Intelligibility (communication)|intelligibility]]''.'''<ref>{{cite book|last1=Taylor|first1=Paul|title=Text-to-speech synthesis|url=https://archive.org/details/texttospeechsynt00tayl_030|url-access=limited|date=2009|publisher=Cambridge University Press|location=Cambridge, UK|isbn=9780521899277|page=[https://archive.org/details/texttospeechsynt00tayl_030/page/n26 3]}}</ref>''' Naturalness describes how closely the output sounds like human speech, while intelligibility is the ease with which the output is understood. The ideal speech synthesizer is both natural and intelligible. Speech synthesis systems usually try to maximize both characteristics.

The two primary technologies generating synthetic speech waveforms are ''concatenative synthesis'' and ''[[formant]] synthesis''. Each technology has strengths and weaknesses, and the intended uses of a synthesis system will typically determine which approach is used.

=== Concatenation synthesis ===
{{main|Concatenative synthesis}}
Concatenative synthesis is based on the concatenation (stringing together) of segments of recorded speech. Generally, concatenative synthesis produces the most natural-sounding synthesized speech. However, differences between natural variations in speech and the nature of the automated techniques for segmenting the waveforms sometimes result in audible glitches in the output. There are three main sub-types of concatenative synthesis.

==== Unit selection synthesis ====

Unit selection synthesis uses large databases of recorded speech. During database creation, each recorded utterance is segmented into some or all of the following: individual [[phone (phonetics)|phones]], [[diphone]]s, half-phones, [[syllable]]s, [[morpheme]]s, [[word]]s, [[phrase]]s, and [[sentence (linguistics)|sentence]]s. Typically, the division into segments is done using a specially modified [[speech recognition|speech recognizer]] set to a "forced alignment" mode with some manual correction afterward, using visual representations such as the [[waveform]] and [[spectrogram]].<ref>[[Alan W. Black]], [https://www.cs.cmu.edu/~awb/papers/IEEE2002/allthetime/allthetime.html Perfect synthesis for all of the people all of the time.] IEEE TTS Workshop 2002.</ref> An [[index (database)|index]] of the units in the speech database is then created based on the segmentation and acoustic parameters like the [[fundamental frequency]] ([[pitch (music)|pitch]]), duration, position in the syllable, and neighboring phones. At [[Run time (program lifecycle phase)|run time]], the desired target utterance is created by determining the best chain of candidate units from the database (unit selection). This process is typically achieved using a specially weighted [[decision tree]].

Unit selection provides the greatest naturalness, because it applies only a small amount of [[digital signal processing]] (DSP) to the recorded speech. DSP often makes recorded speech sound less natural, although some systems use a small amount of signal processing at the point of concatenation to smooth the waveform. The output from the best unit-selection systems is often indistinguishable from real human voices, especially in contexts for which the TTS system has been tuned. However, maximum naturalness typically require unit-selection speech databases to be very large, in some systems ranging into the [[gigabyte]]s of recorded data, representing dozens of hours of speech.<ref>John Kominek and [[Alan W. Black]]. (2003). CMU ARCTIC databases for speech synthesis. CMU-LTI-03-177. Language Technologies Institute, School of Computer Science, Carnegie Mellon University.</ref> Also, unit selection algorithms have been known to select segments from a place that results in less than ideal synthesis (e.g. minor words become unclear) even when a better choice exists in the database.<ref>Julia Zhang. [http://groups.csail.mit.edu/sls/publications/2004/zhang_thesis.pdf Language Generation and Speech Synthesis in Dialogues for Language Learning], masters thesis, Section 5.6 on page 54.</ref> Recently, researchers have proposed various automated methods to detect unnatural segments in unit-selection speech synthesis systems.<ref>William Yang Wang and Kallirroi Georgila. (2011). [https://www.cs.cmu.edu/~yww/papers/asru2011.pdf Automatic Detection of Unnatural Word-Level Segments in Unit-Selection Speech Synthesis], IEEE ASRU 2011.</ref>

==== Diphone synthesis ====

Diphone synthesis uses a minimal speech database containing all the [[diphone]]s (sound-to-sound transitions) occurring in a language. The number of diphones depends on the [[phonotactics]] of the language: for example, Spanish has about 800 diphones, and German about 2500. In diphone synthesis, only one example of each diphone is contained in the speech database. At runtime, the target [[prosody (linguistics)|prosody]] of a sentence is superimposed on these minimal units by means of [[digital signal processing]] techniques such as [[linear predictive coding]], [[PSOLA]]<ref>{{cite web|title=Pitch-Synchronous Overlap and Add (PSOLA) Synthesis|url=http://www.fon.hum.uva.nl/praat/manual/PSOLA.html|url-status=dead|archive-url=https://web.archive.org/web/20070222180903/http://www.fon.hum.uva.nl/praat/manual/PSOLA.html|archive-date=February 22, 2007|access-date=2008-05-28}}</ref> or [[MBROLA]].<ref>T. Dutoit, V. Pagel, N. Pierret, F. Bataille, O. van der Vrecken. [http://ai2-s2-pdfs.s3.amazonaws.com/7b1f/dadf05b8f968a5b361f6f82852ade62c8010.pdf The MBROLA Project: Towards a set of high quality speech synthesizers of use for non commercial purposes]. ''ICSLP Proceedings'', 1996.</ref> or more recent techniques such as pitch modification in the source domain using [[discrete cosine transform]].<ref name="Muralishankar2004">{{cite journal | last1 = Muralishankar | first1 = R. | last2 = Ramakrishnan | first2 = A. G. | last3 = Prathibha | first3 = P. | date = February 2004 | title = Modification of Pitch using DCT in the Source Domain | journal = Speech Communication | volume = 42 | issue = 2 | pages = 143–154 | doi=10.1016/j.specom.2003.05.001}}</ref> Diphone synthesis suffers from the sonic glitches of concatenative synthesis and the robotic-sounding nature of formant synthesis, and has few of the advantages of either approach other than small size. As such, its use in commercial applications is declining,{{Citation needed|date=January 2012}} although it continues to be used in research because there are a number of freely available software implementations. An early example of Diphone synthesis is a teaching robot, [[Leachim (Robot)|Leachim]], that was invented by [[Michael J. Freeman]].<ref>{{Cite news|url=http://content.time.com/time/magazine/article/0,9171,904056,00.html|title=Education: Marvel of The Bronx|date=1974-04-01|magazine=Time|access-date=2019-05-28|language=en-US|issn=0040-781X}}</ref> Leachim contained information regarding class curricular and certain biographical information about the students whom it was programmed to teach.<ref>{{Cite web|url=http://cyberneticzoo.com/robots/1960-rudy-the-robot-michael-freeman-american/|title=1960 - Rudy the Robot - Michael Freeman (American)|date=2010-09-13|website=cyberneticzoo.com|language=en-US|access-date=2019-05-23}}</ref> It was tested in a fourth grade classroom in [[The Bronx|the Bronx, New York]].<ref>{{Cite book|url=https://books.google.com/books?id=bNECAAAAMBAJ&q=Leachim+Michael+Freeman&pg=PA40|title=New York Magazine|date=1979-07-30|publisher=New York Media, LLC|language=en}}</ref><ref>{{Cite book|url=https://books.google.com/books?id=_QJmAAAAMAAJ&q=leachim|title=The Futurist|date=1978|publisher=World Future Society.|pages=359, 360, 361|language=en}}</ref>

==== Domain-specific synthesis ====

Domain-specific synthesis concatenates prerecorded words and phrases to create complete utterances. It is used in applications where the variety of texts the system will output is limited to a particular domain, like transit schedule announcements or weather reports.<ref>[[Lori Lamel|L.F. Lamel]], J.L. Gauvain, B. Prouts, C. Bouhier, R. Boesch. [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.53.6101&rep=rep1&type=pdf Generation and Synthesis of Broadcast Messages], ''Proceedings ESCA-NATO Workshop and Applications of Speech Technology'', September 1993.</ref> The technology is very simple to implement, and has been in commercial use for a long time, in devices like talking clocks and calculators. The level of naturalness of these systems can be very high because the variety of sentence types is limited, and they closely match the prosody and intonation of the original recordings.{{Citation needed|date=February 2007}}

Because these systems are limited by the words and phrases in their databases, they are not general-purpose and can only synthesize the combinations of words and phrases with which they have been preprogrammed. The blending of words within naturally spoken language however can still cause problems unless the many variations are taken into account. For example, in [[rhotic and non-rhotic accents|non-rhotic]] dialects of English the ''"r"'' in words like ''"clear"'' {{IPA|/ˈklɪə/}} is usually only pronounced when the following word has a vowel as its first letter (e.g. ''"clear out"'' is realized as {{IPA|/ˌklɪəɹˈʌʊt/}}). Likewise in [[French language|French]], many final consonants become no longer silent if followed by a word that begins with a vowel, an effect called [[Liaison (French)|liaison]]. This [[alternation (linguistics)|alternation]] cannot be reproduced by a simple word-concatenation system, which would require additional complexity to be [[context-sensitive grammar|context-sensitive]].

=== Formant synthesis ===

[[Formant]] synthesis does not use human speech samples at runtime. Instead, the synthesized speech output is created using [[additive synthesis]] and an acoustic model ([[physical modelling synthesis]]).<ref>Dartmouth College: [http://digitalmusics.dartmouth.edu/~book/MATCpages/chap.4/4.4.formant_synth.html ''Music and Computers''] {{webarchive|url=https://web.archive.org/web/20110608035309/http://digitalmusics.dartmouth.edu/~book/MATCpages/chap.4/4.4.formant_synth.html |date=2011-06-08 }}, 1993.</ref> Parameters such as [[fundamental frequency]], [[phonation|voicing]], and [[noise]] levels are varied over time to create a [[waveform]] of artificial speech. This method is sometimes called ''rules-based synthesis''; however, many concatenative systems also have rules-based components.
Many systems based on formant synthesis technology generate artificial, robotic-sounding speech that would never be mistaken for human speech. However, maximum naturalness is not always the goal of a speech synthesis system, and formant synthesis systems have advantages over concatenative systems. Formant-synthesized speech can be reliably intelligible, even at very high speeds, avoiding the acoustic glitches that commonly plague concatenative systems. High-speed synthesized speech is used by the visually impaired to quickly navigate computers using a [[screen reader]]. Formant synthesizers are usually smaller programs than concatenative systems because they do not have a database of speech samples. They can therefore be used in [[embedded system]]s, where [[data storage device|memory]] and [[microprocessor]] power are especially limited. Because formant-based systems have complete control of all aspects of the output speech, a wide variety of prosodies and [[intonation (linguistics)|intonation]]s can be output, conveying not just questions and statements, but a variety of emotions and tones of voice.

Examples of non-real-time but highly accurate intonation control in formant synthesis include the work done in the late 1970s for the [[Texas Instruments]] toy [[Speak & Spell (game)|Speak & Spell]], and in the early 1980s [[Sega]] [[Video arcade|arcade]] machines<ref>Examples include [[Astro Blaster]], [[Space Fury]], and [[Star Trek (arcade game)|Star Trek: Strategic Operations Simulator]]</ref> and in many [[Atari, Inc.]] arcade games<ref>Examples include [[Star Wars (arcade game)|Star Wars]], [[Firefox (arcade game)|Firefox]], [[Star Wars: Return of the Jedi (arcade game)|Return of the Jedi]], [[Road Runner (video game)|Road Runner]], [[Star Wars: The Empire Strikes Back (arcade game)|The Empire Strikes Back]], [[Indiana Jones and the Temple of Doom (arcade game)|Indiana Jones and the Temple of Doom]], [[720°]], [[Gauntlet (arcade game)|Gauntlet]], [[Gauntlet II]], [[A.P.B. (video game)|A.P.B.]], [[Paperboy (video game)|Paperboy]], [[RoadBlasters]],  [http://www.arcade-museum.com/game_detail.php?game_id=10319 Vindicators Part II], [[Escape from the Planet of the Robot Monsters]].</ref> using the [[Texas Instruments LPC Speech Chips|TMS5220 LPC Chips]]. Creating proper intonation for these projects was painstaking, and the results have yet to be matched by real-time text-to-speech interfaces.<ref>{{Cite book |author=John Holmes and Wendy Holmes |title=Speech Synthesis and Recognition |edition=2nd |publisher=CRC |year=2001 |isbn=978-0-7484-0856-6}}</ref>{{When|date=April 2025}}

=== Articulatory synthesis ===
{{Main|Articulatory synthesis}}
Articulatory synthesis consists of computational techniques for synthesizing speech based on models of the human [[vocal tract]] and the articulation processes occurring there. The first articulatory synthesizer regularly used for laboratory experiments was developed at [[Haskins Laboratories]] in the mid-1970s by [[Philip Rubin]], Tom Baer, and Paul Mermelstein. This synthesizer, known as ASY, was based on vocal tract models developed at [[Bell Laboratories]] in the 1960s and 1970s by Paul Mermelstein, Cecil Coker, and colleagues.

Until recently, articulatory synthesis models have not been incorporated into commercial speech synthesis systems. A notable exception is the [[NeXT]]-based system originally developed and marketed by Trillium Sound Research, a spin-off company of the [[University of Calgary]], where much of the original research was conducted. Following the demise of the various incarnations of NeXT (started by [[Steve Jobs]] in the late 1980s and merged with Apple Computer in 1997), the Trillium software was published under the GNU General Public License, with work continuing as [[gnuspeech]]. The system, first marketed in 1994, provides full articulatory-based text-to-speech conversion using a waveguide or transmission-line analog of the human oral and nasal tracts controlled by Carré's "distinctive region model".

More recent synthesizers, developed by Jorge C. Lucero and colleagues, incorporate models of vocal fold biomechanics, glottal aerodynamics and acoustic wave propagation in the bronchi, trachea, nasal and oral cavities, and thus constitute full systems of physics-based speech simulation.<ref name=":0">{{Cite journal|url = http://www.cic.unb.br/~lucero/papers/768_Paper.pdf|title = Physics-based synthesis of disordered voices|last1 = Lucero|first1 = J. C.|date = 2013|journal = Interspeech 2013|access-date = Aug 27, 2015|last2 = Schoentgen|first2 = J.|last3 = Behlau|first3 = M.|pages = 587–591|publisher = International Speech Communication Association|location = Lyon, France|doi = 10.21437/Interspeech.2013-161| s2cid=17451802 }}</ref><ref name=":1">{{Cite journal|last1=Englert|first1=Marina|last2=Madazio|first2=Glaucya|last3=Gielow|first3=Ingrid|last4=Lucero|first4=Jorge|last5=Behlau|first5=Mara|date=2016|title=Perceptual error identification of human and synthesized voices|journal=Journal of Voice|volume=30|issue=5|pages=639.e17–639.e23|doi=10.1016/j.jvoice.2015.07.017|pmid=26337775}}</ref>

=== HMM-based synthesis ===

HMM-based synthesis is a synthesis method based on [[hidden Markov model]]s, also called Statistical Parametric Synthesis. In this system, the [[frequency spectrum]] ([[vocal tract]]), [[fundamental frequency]] (voice source), and duration ([[prosody (linguistics)|prosody]]) of speech are modeled simultaneously by HMMs. Speech [[waveforms]] are generated from HMMs themselves based on the [[maximum likelihood]] criterion.<ref>{{cite web |url=http://hts.sp.nitech.ac.jp/ |title=The HMM-based Speech Synthesis System |publisher=Hts.sp.nitech.ac.j |access-date=2012-02-22 |archive-date=2012-02-13 |archive-url=https://web.archive.org/web/20120213232606/http://hts.sp.nitech.ac.jp/ |url-status=dead }}</ref>

=== Sinewave synthesis ===
{{Main|Sinewave synthesis}}
Sinewave synthesis is a technique for synthesizing speech by replacing the [[formants]] (main bands of energy) with pure tone whistles.<ref>{{Cite journal
 |last1        = Remez
 |first1       = R.
 |last2        = Rubin
 |first2       = P.
 |last3        = Pisoni
 |first3       = D.
 |last4        = Carrell
 |first4       = T.
 |title        = Speech perception without traditional speech cues
 |doi          = 10.1126/science.7233191
 |journal      = Science
 |volume       = 212
 |issue        = 4497
 |pages        = 947–949
 |date         = 22 May 1981
 |pmid         = 7233191
 |bibcode = 1981Sci...212..947R
 |url          = http://www.bsos.umd.edu/hesp/mwinn/Remez_et_al_1981.pdf
 |access-date  = 2011-12-14
 |archive-url  = https://web.archive.org/web/20111216113028/http://www.bsos.umd.edu/hesp/mwinn/Remez_et_al_1981.pdf
 |archive-date = 2011-12-16
 |url-status     = dead
}}<!-- in case PDF link dies, paper also available here and here:
http://people.ece.cornell.edu/land/courses/ece4760/Speech/remez_rubin_pisoni_carrell1981.pdf
http://www.haskins.yale.edu/Reprints/HL0338.pdf --></ref>

=== Deep learning-based synthesis ===
{{Main|Deep learning speech synthesis}}
[[File:Larynx-HiFi-GAN speech sample.wav|thumb|Speech synthesis example using the HiFi-GAN neural vocoder]]
Deep learning speech synthesis uses [[deep neural network]]s (DNN) to produce artificial speech from text (text-to-speech) or spectrum (vocoder).
The deep neural networks are trained using a large amount of recorded speech and, in the case of a text-to-speech system, the associated labels and/or input text.

[[15.ai]] uses a ''multi-speaker model''—hundreds of voices are trained concurrently rather than sequentially, decreasing the required training time and enabling the model to learn and generalize shared emotional context, even for voices with no exposure to such emotional context.<ref>{{cite web |last=Temitope |first=Yusuf |date=December 10, 2024 |title=15.ai Creator reveals journey from MIT Project to internet phenomenon |url=https://guardian.ng/technology/15-ai-creator-reveals-journey-from-mit-project-to-internet-phenomenon/ |access-date=December 25, 2024 |website=[[The Guardian (Nigeria)|The Guardian]] |quote= |archive-url=https://web.archive.org/web/20241228152312/https://guardian.ng/technology/15-ai-creator-reveals-journey-from-mit-project-to-internet-phenomenon/ |archive-date=December 28, 2024}}</ref> The [[deep learning]] model used by the application is [[Nondeterministic algorithm|nondeterministic]]: each time that speech is generated from the same string of text, the intonation of the speech will be slightly different. The application also supports manually altering the [[Emotional prosody|emotion]] of a generated line using ''emotional contextualizers'' (a term coined by this project), a sentence or phrase that conveys the emotion of the take that serves as a guide for the model during inference.<ref name="automaton2">{{cite web |last=Kurosawa |first=Yuki |date=2021-01-19 |title=ゲームキャラ音声読み上げソフト「15.ai」公開中。『Undertale』や『Portal』のキャラに好きなセリフを言ってもらえる |url=https://automaton-media.com/articles/newsjp/20210119-149494/ |url-status=live |archive-url=https://web.archive.org/web/20210119103031/https://automaton-media.com/articles/newsjp/20210119-149494/ |archive-date=2021-01-19 |access-date=2021-01-19 |website=AUTOMATON |quote=}}</ref><ref name="Denfaminicogamer2">{{cite web |last=Yoshiyuki |first=Furushima |date=2021-01-18 |title=『Portal』のGLaDOSや『UNDERTALE』のサンズがテキストを読み上げてくれる。文章に込められた感情まで再現することを目指すサービス「15.ai」が話題に |url=https://news.denfaminicogamer.jp/news/210118f |url-status=live |archive-url=https://web.archive.org/web/20210118051321/https://news.denfaminicogamer.jp/news/210118f |archive-date=2021-01-18 |access-date=2021-01-18 |website=Denfaminicogamer |quote=}}</ref>

[[ElevenLabs]] is primarily known for its [[browser-based]], AI-assisted text-to-speech software, Speech Synthesis, which can produce lifelike speech by synthesizing [[vocal emotion]] and [[Intonation (linguistics)|intonation]].<ref>{{Cite web |date=January 23, 2023 |title=Generative AI comes for cinema dubbing: Audio AI startup ElevenLabs raises pre-seed |url=https://sifted.eu/articles/generative-ai-audio-elevenlabs/ |access-date=2023-02-03 |website=Sifted |language=en-US}}</ref> The company states its software is built to adjust the intonation and pacing of delivery based on the context of language input used.<ref name=":13">{{Cite magazine |last=Ashworth |first=Boone |date=April 12, 2023 |title=AI Can Clone Your Favorite Podcast Host's Voice |url=https://www.wired.com/story/ai-podcasts-podcastle-revoice-descript/ |magazine=Wired |language=en-US |access-date=2023-04-25}}</ref> It uses advanced algorithms to analyze the contextual aspects of text, aiming to detect emotions like anger, sadness, happiness, or alarm, which enables the system to understand the user's sentiment,<ref>{{Cite magazine |author=WIRED Staff |title=This Podcast Is Not Hosted by AI Voice Clones. We Swear |url=https://www.wired.com/story/gadget-lab-podcast-594/ |magazine=Wired |language=en-US |issn=1059-1028 |access-date=2023-07-25}}</ref> resulting in a more realistic and human-like inflection. Other features include multilingual speech generation and long-form content creation with contextually-aware voices.<ref name=":34">{{Cite web |last=Wiggers |first=Kyle |date=2023-06-20 |title=Voice-generating platform ElevenLabs raises $19M, launches detection tool |url=https://techcrunch.com/2023/06/20/voice-generating-platform-elevenlabs-raises-19m-launches-detection-tool/ |access-date=2023-07-25 |website=TechCrunch |language=en-US}}</ref><ref>{{Cite web |last=Bonk |first=Lawrence |title=ElevenLabs' Powerful New AI Tool Lets You Make a Full Audiobook in Minutes |url=https://www.lifewire.com/elevenlabs-new-audiobook-ai-tool-7550061 |access-date=2023-07-25 |website=Lifewire |language=en}}</ref>

The DNN-based speech synthesizers are approaching the naturalness of the human voice.
Examples of disadvantages of the method are low robustness when the data are not sufficient, lack of controllability and low performance in auto-regressive models.

For tonal languages, such as Chinese or Taiwanese language, there are different levels of [[tone sandhi]] required and sometimes the output of speech synthesizer may result in the mistakes of tone sandhi.<ref>{{Cite journal |last=Zhu |first=Jian |date=2020-05-25 |title=Probing the phonetic and phonological knowledge of tones in Mandarin TTS models |url=http://dx.doi.org/10.21437/speechprosody.2020-190 |journal=Speech Prosody 2020 |pages=930–934 |location=ISCA |publisher=ISCA |doi=10.21437/speechprosody.2020-190|arxiv=1912.10915 |s2cid=209444942 }}</ref>

=== Audio deepfakes ===
{{excerpt|Audio deepfake}}In 2023, [[Vice Media|VICE]] reporter [[Joseph Cox (journalist)|Joseph Cox]] published findings that he had recorded five minutes of himself talking and then used a tool developed by ElevenLabs to create voice deepfakes that defeated a bank's [[Speaker recognition|voice-authentication]] system.<ref>{{Cite magazine |last=Newman |first=Lily Hay |title=AI-Generated Voice Deepfakes Aren't Scary Good—Yet |url=https://www.wired.com/story/ai-voice-deep-fakes/ |magazine=Wired |language=en-US |issn=1059-1028 |access-date=2023-07-25}}</ref>

== Challenges ==

=== Text normalization challenges ===

The process of normalizing text is rarely straightforward. Texts are full of [[heteronym (linguistics)|heteronym]]s, [[number]]s, and [[abbreviation]]s that all require expansion into a phonetic representation. There are many spellings in English which are pronounced differently based on context. For example, "My latest project is to learn how to better project my voice" contains two pronunciations of "project".

Most text-to-speech (TTS) systems do not generate [[Semantics#Linguistics|semantic]] representations of their input texts, as processes for doing so are unreliable, poorly understood, and computationally ineffective. As a result, various [[heuristic]] techniques are used to guess the proper way to disambiguate [[homograph]]s, like examining neighboring words and using statistics about frequency of occurrence.

Recently TTS systems have begun to use HMMs (discussed [[Speech synthesis#HMM-based synthesis|above]]) to generate "[[Part-of-speech tagging|parts of speech]]" to aid in disambiguating homographs. This technique is quite successful for many cases such as whether "read" should be pronounced as "red" implying past tense, or as "reed" implying present tense. Typical error rates when using HMMs in this fashion are usually below five percent. These techniques also work well for most European languages, although access to required training [[Text corpus|corpora]] is frequently difficult in these languages.

Deciding how to convert numbers is another problem that TTS systems have to address. It is a simple programming challenge to convert a number into words (at least in English), like "1325" becoming "one thousand three hundred twenty-five". However, numbers occur in many different contexts; "1325" may also be read as "one three two five", "thirteen twenty-five" or "thirteen hundred and twenty five". A TTS system can often infer how to expand a number based on surrounding words, numbers, and punctuation, and sometimes the system provides a way to specify the context if it is ambiguous.<ref>{{cite web | title = Speech synthesis | publisher = World Wide Web Organization | url = http://www.w3.org/TR/speech-synthesis/#S3.1.8}}</ref> Roman numerals can also be read differently depending on context. For example, "Henry VIII" reads as "Henry the Eighth", while "Chapter VIII" reads as "Chapter Eight".

Similarly, abbreviations can be ambiguous. For example, the abbreviation "in" for "inches" must be differentiated from the word "in", and the address "12 St John St." uses the same abbreviation for both "Saint" and "Street". TTS systems with intelligent front ends can make educated guesses about ambiguous abbreviations, while others provide the same result in all cases, resulting in nonsensical (and sometimes comical) outputs, such as "[[Ulysses S. Grant]]" being rendered as "Ulysses South Grant".

=== Text-to-phoneme challenges ===

{{Unreferenced section|date=April 2023}}

Speech synthesis systems use two basic approaches to determine the pronunciation of a word based on its [[spelling]], a process which is often called text-to-phoneme or [[grapheme]]-to-phoneme conversion ([[phoneme]] is the term used by [[Linguistics|linguists]] to describe distinctive sounds in a [[language]]). The simplest approach to text-to-phoneme conversion is the dictionary-based approach, where a large dictionary containing all the words of a language and their correct [[pronunciation]]s is stored by the program. Determining the correct pronunciation of each word is a matter of looking up each word in the dictionary and replacing the spelling with the pronunciation specified in the dictionary. The other approach is rule-based, in which pronunciation rules are applied to words to determine their pronunciations based on their spellings. This is similar to the "sounding out", or [[synthetic phonics]], approach to learning reading.

Each approach has advantages and drawbacks. The dictionary-based approach is quick and accurate, but completely fails if it is given a word which is not in its dictionary. As dictionary size grows, so too does the memory space requirements of the synthesis system. On the other hand, the rule-based approach works on any input, but the complexity of the rules grows substantially as the system takes into account irregular spellings or pronunciations. (Consider that the word "of" is very common in English, yet is the only word in which the letter "f" is pronounced {{IPA|[v]}}.) As a result, nearly all speech synthesis systems use a combination of these approaches.

Languages with a [[phonemic orthography]] have a very regular writing system, and the prediction of the pronunciation of words based on their spellings is quite successful. Speech synthesis systems for such languages often use the rule-based method extensively, resorting to dictionaries only for those few words, like foreign names and loanwords, whose pronunciations are not obvious from their spellings. On the other hand, speech synthesis systems for languages like English, which have extremely irregular spelling systems, are more likely to rely on dictionaries, and to use rule-based methods only for unusual words, or words that are not in their dictionaries.

=== Evaluation challenges ===
The consistent evaluation of speech synthesis systems may be difficult because of a lack of universally agreed objective evaluation criteria. Different organizations often use different speech data. The quality of speech synthesis systems also depends on the quality of the production technique (which may involve analogue or digital recording) and on the facilities used to replay the speech. Evaluating speech synthesis systems has therefore often been compromised by differences between production techniques and replay facilities.

Since 2005, however, some researchers have started to evaluate speech synthesis systems using a common speech dataset.<ref>{{cite web|url=http://festvox.org/blizzard |title=Blizzard Challenge |publisher=Festvox.org |access-date=2012-02-22}}</ref>

=== Prosodics and emotional content ===
{{See also|Emotional speech recognition|Prosody (linguistics)}}
A study in the journal ''Speech Communication'' by Amy Drahota and colleagues at the [[University of Portsmouth]], [[UK]], reported that listeners to voice recordings could determine, at better than chance levels, whether or not the speaker was smiling.<ref>{{Cite news|title=Smile -and the world can hear you |date=January 9, 2008 |url=http://www.port.ac.uk/aboutus/newsandevents/news/title,74220,en.html |archive-date=May 17, 2008 |archive-url=https://web.archive.org/web/20080517102201/http://www.port.ac.uk/aboutus/newsandevents/news/title%2C74220%2Cen.html |publisher=University of Portsmouth |url-status=dead }}</ref><ref>{{Cite news |title=Smile – And The World Can Hear You, Even If You Hide |work=Science Daily |date=January 2008 |url=https://www.sciencedaily.com/releases/2008/01/080111224745.htm}}</ref><ref>{{Cite journal
 |last1       = Drahota
 |first1      = A.
 |title       = The vocal communication of different kinds of smile
 |doi         = 10.1016/j.specom.2007.10.001
 |journal     = Speech Communication
 |volume      = 50
 |issue       = 4
 |pages       = 278–287
 |year        = 2008
 |s2cid = 46693018
 |url         = http://peer.ccsd.cnrs.fr/docs/00/49/91/97/PDF/PEER_stage2_10.1016%252Fj.specom.2007.10.001.pdf
 |url-status     = dead
 |archive-url  = https://web.archive.org/web/20130703062330/https://peer.ccsd.cnrs.fr/docs/00/49/91/97/PDF/PEER_stage2_10.1016/j.specom.2007.10.001.pdf
 |archive-date = 2013-07-03
}}<!-- also available here: http://ganymedes.lib.unideb.hu:8080/udpeer/bitstream/2437.2/2984/1/PEER_stage2_10.1016%252Fj.specom.2007.10.001.pdf --></ref> It was suggested that identification of the vocal features that signal emotional content may be used to help make synthesized speech sound more natural. One of the related issues is modification of the [[pitch contour]] of the sentence, depending upon whether it is an affirmative, interrogative or exclamatory sentence. One of the techniques for pitch modification<ref name="Muralishankar2004" /> uses [[discrete cosine transform]] in the source domain ([[linear prediction]] residual). Such pitch synchronous pitch modification techniques need a priori pitch marking of the synthesis speech database using techniques such as epoch extraction using dynamic [[Plosive|plosion]] index applied on the integrated linear prediction residual of the [[Voice (phonetics)|voiced]] regions of speech.<ref>{{cite journal|last1=Prathosh|first1=A. P.|last2=Ramakrishnan|first2=A. G.|last3=Ananthapadmanabha|first3=T. V.|title=Epoch extraction based on integrated linear prediction residual using plosion index|journal=IEEE Trans. Audio Speech Language Processing|date=December 2013|volume=21|issue=12|pages=2471–2480|doi=10.1109/TASL.2013.2273717|s2cid=10491251}}</ref>  In general, prosody remains a challenge for speech synthesizers, and is an active research topic.

== Dedicated hardware ==
[[File:Electronic_Vowel_Synthesizer,_Bell_System_Science_Experiment_No.3.jpg|thumb|A speech synthesis kit produced by Bell System]]
* [[Icophone]]
* [[General Instrument SP0256-AL2]]
* [[National Semiconductor]] DT1050 Digitalker (Mozer – [[Forrest Mozer]])
* [[Texas Instruments LPC Speech Chips]]<ref name="TI will exit dedicated speech-synthesis chips, transfer products to Sensory">EE Times. "[http://www.eetimes.com/electronics-news/4102385/TI-will-exit-dedicated-speech-synthesis-chips-transfer-products-to-Sensory TI will exit dedicated speech-synthesis chips, transfer products to Sensory] {{Webarchive|url=https://web.archive.org/web/20120528014257/http://www.eetimes.com/electronics-news/4102385/TI-will-exit-dedicated-speech-synthesis-chips-transfer-products-to-Sensory |date=2012-05-28 }}." June 14, 2001.</ref>

==Hardware and software systems==
Popular systems offering speech synthesis as a built-in capability.

=== Texas Instruments ===
{{main article|Texas Instruments LPC Speech Chips}}
[[File:Texas Instruments TI-99 4A speech demo.flac|thumb|TI-99/4A speech demo using the built-in vocabulary]]

In the early 1980s, TI was known as a pioneer in speech synthesis, and a highly popular plug-in speech synthesizer module was available for the TI-99/4 and 4A. Speech synthesizers were offered free with the purchase of a number of cartridges and were used by many TI-written video games (games offered with speech during this promotion included ''[[Alpiner (video game)|Alpiner]]'' and ''[[Parsec (video game)|Parsec]]''). The synthesizer uses a variant of linear predictive coding and has a small in-built vocabulary. The original intent was to release small cartridges that plugged directly into the synthesizer unit, which would increase the device's built-in vocabulary. However, the success of software text-to-speech in the Terminal Emulator II cartridge canceled that plan.

===Mattel===
The [[Mattel]] [[Intellivision]] game console offered the [[Intellivoice]] Voice Synthesis module in 1982. It included the [[General Instrument SP0256|SP0256 Narrator]] speech synthesizer chip on a removable cartridge. The Narrator had 2kB of Read-Only Memory (ROM), and this was utilized to store a database of generic words that could be combined to make phrases in Intellivision games. Since the Orator chip could also accept speech data from external memory, any additional words or phrases needed could be stored inside the cartridge itself. The data consisted of strings of analog-filter coefficients to modify the behavior of the chip's synthetic vocal-tract model, rather than simple digitized samples.

===SAM===
[[File:C64 Software Automatic Mouth demo.flac|thumb|A demo of SAM on the C64]]
Also released in 1982, [[Software Automatic Mouth]] was the first commercial all-software voice synthesis program. It was later used as the basis for [[Macintalk]]. The program was available for non-Macintosh Apple computers (including the Apple II, and the Lisa), various Atari models and the Commodore 64. The Apple version preferred additional hardware that contained DACs, although it could instead use the computer's one-bit audio output (with the addition of much distortion) if the card was not present. The Atari made use of the embedded POKEY audio chip. Speech playback on the Atari normally disabled interrupt requests and shut down the ANTIC chip during vocal output. The audible output is extremely distorted speech when the screen is on. The Commodore 64 made use of the 64's embedded SID audio chip.

=== Atari ===
[[File:Atari ST speech synthesis demo.flac|thumb|Atari ST speech synthesis demo]]

Arguably, the first speech system integrated into an [[operating system]] was the circa 1983 unreleased Atari [[Atari 8-bit computers#Unreleased XL models|1400XL/1450XL]] computers. These used the Votrax SC01 chip and a [[finite-state machine]] to enable World English Spelling text-to-speech synthesis.<ref>{{cite web |url=http://www.atarimuseum.com/ahs_archives/archives/pdf/computers/8bits/1400xlmodem.pdf |title=1400XL/1450XL Speech Handler External Reference Specification |access-date=2012-02-22 |archive-date=2012-03-24 |archive-url=https://web.archive.org/web/20120324014644/http://www.atarimuseum.com/ahs_archives/archives/pdf/computers/8bits/1400xlmodem.pdf |url-status=dead }}</ref>

The [[Atari ST]] computers were sold with "stspeech.tos" on floppy disk.

=== Apple ===
[[File:MacinTalk 1 demo.flac|thumb|MacinTalk 1 demo]]
[[File:MacinTalk 2 demo.flac|thumb|MacinTalk 2 demo featuring the Mr. Hughes and Marvin voices]]
The first speech system integrated into an [[operating system]] that shipped in quantity was [[Apple Computer]]'s [[PlainTalk#Original MacInTalk|MacInTalk]]. The software was licensed from third-party developers Joseph Katz and Mark Barton (later, SoftVoice, Inc.) and was featured during the 1984 introduction of the Macintosh computer. This January demo required 512 kilobytes of RAM memory. As a result, it could not run in the 128 kilobytes of RAM the first Mac actually shipped with.<ref name="demo">{{cite web|url=http://www.folklore.org/StoryView.py?story=Intro_Demo.txt |title=It Sure Is Great To Get Out Of That Bag! |publisher=folklore.org |access-date=2013-03-24}}</ref> So, the demo was accomplished with a prototype 512k Mac, although those in attendance were not told of this and the synthesis demo created considerable excitement for the Macintosh. In the early 1990s Apple expanded its capabilities offering system wide text-to-speech support. With the introduction of faster PowerPC-based computers they included higher quality voice sampling. Apple also introduced [[speech recognition]] into its systems which provided a fluid command set. More recently, Apple has added sample-based voices. Starting as a curiosity, the speech system of Apple [[Macintosh]] has evolved into a fully supported program, [[PlainTalk]], for people with vision problems. [[VoiceOver]] was for the first time featured in 2005 in [[Mac OS X Tiger]] (10.4). During 10.4 (Tiger) and first releases of 10.5 ([[Mac OS X Leopard|Leopard]]) there was only one standard voice shipping with Mac OS X. Starting with 10.6 ([[Mac OS X Snow Leopard|Snow Leopard]]), the user can choose out of a wide range list of multiple voices. VoiceOver voices feature the taking of realistic-sounding breaths between sentences, as well as improved clarity at high read rates over PlainTalk. Mac OS X also includes [[say (software)|say]], a [[Command-line interface|command-line based]] application that converts text to audible speech. The [[AppleScript]] Standard Additions includes a say verb that allows a script to use any of the installed voices and to control the pitch, speaking rate and modulation of the spoken text.

=== Amazon ===
Used in [[Amazon Alexa|Alexa]] and as [[Software as a service|Software as a Service]] in AWS<ref>{{Cite web|url=https://aws.amazon.com/polly/|title=Amazon Polly|website=Amazon Web Services, Inc.|language=en-US|access-date=2020-04-28}}</ref> (from 2017).

=== AmigaOS ===
[[File:Amiga speech synthesis.flac|thumb|Example of speech synthesis with the included Say utility in Workbench 1.3]]
[[File:SoftVoice.svg|right|upright]]
The second operating system to feature advanced speech synthesis capabilities was [[AmigaOS]], introduced in 1985. The voice synthesis was licensed by [[Commodore International]] from SoftVoice, Inc., who also developed the original [[MacinTalk]] text-to-speech system. It featured a complete system of voice emulation for American English, with both male and female voices and "stress" indicator markers, made possible through the [[Amiga]]'s audio [[chipset]].<ref>{{Cite book |author=Miner, Jay |year=1991 |title=Amiga Hardware Reference Manual |edition=3rd |publisher=[[Addison-Wesley]] Publishing Company, Inc. |isbn=978-0-201-56776-2|display-authors=etal|author-link=Jay Miner }}</ref> The synthesis system was divided into a translator library which converted unrestricted English text into a standard set of phonetic codes and a narrator device which implemented a formant model of speech generation.. AmigaOS also featured a high-level "[[AmigaOS#Speech synthesis|Speak Handler]]", which allowed command-line users to redirect text output to speech. Speech synthesis was occasionally used in third-party programs, particularly word processors and educational software. The synthesis software remained largely unchanged from the first AmigaOS release and Commodore eventually removed speech synthesis support from AmigaOS 2.1 onward.

Despite the American English phoneme limitation, an unofficial version with multilingual speech synthesis was developed. This made use of an enhanced version of the translator library which could translate a number of languages, given a set of rules for each language.<ref>{{cite web|url=http://uk.aminet.net/util/libs/translator42.readme |title=Translator Library (Multilingual-speech version) |last1=Devitt |first1=Francesco |date=30 June 1995 |access-date=9 April 2013 |url-status=dead |archive-url=https://web.archive.org/web/20120226143859/https://uk.aminet.net/util/libs/translator42.readme |archive-date=26 February 2012 }}</ref>

=== Microsoft Windows ===
{{See also|Microsoft Agent}}

Modern [[Microsoft Windows|Windows]] desktop systems can use [[Speech Application Programming Interface#SAPI 1-4 API family|SAPI 4]] and [[Speech Application Programming Interface#SAPI 5 API family|SAPI 5]] components to support speech synthesis and [[speech recognition]]. SAPI 4.0 was available as an optional add-on for [[Windows 95]] and [[Windows 98]]. [[Windows 2000]] added [[Microsoft Narrator|Narrator]], a text-to-speech utility for people who have visual impairment. Third-party programs such as JAWS for Windows, Window-Eyes, Non-visual Desktop Access, Supernova and System Access can perform various text-to-speech tasks such as reading text aloud from a specified website, email account, text document, the Windows clipboard, the user's keyboard typing, etc. Not all programs can use speech synthesis directly.<ref name="Narrator">{{cite web|date=2011-01-29|title=Accessibility Tutorials for Windows XP: Using Narrator|url=http://www.microsoft.com/enable/training/windowsxp/usingnarrator.aspx|url-status=dead|archive-url=https://web.archive.org/web/20030621002716/http://www.microsoft.com/enable/training/windowsxp/usingnarrator.aspx|archive-date=June 21, 2003|access-date=2011-01-29|publisher=Microsoft}}</ref> Some programs can use plug-ins, extensions or add-ons to read text aloud. Third-party programs are available that can read text from the system clipboard.

[[Microsoft Speech Server]] is a server-based package for voice synthesis and recognition. It is designed for network use with [[web applications]] and [[call centers]].

=== Votrax ===
{{Main article|Votrax}}
[[File:Votrax speech synthesizer demo.flac|thumb|Votrax Type 'N Talk speech synthesizer (1980)]]
From 1971 to 1996, Votrax produced a number of commercial speech synthesizer components.  A Votrax synthesizer was included in the first generation Kurzweil Reading Machine for the Blind.

==Text-to-speech systems==
Text-to-speech (TTS) refers to the ability of computers to read text aloud. A TTS engine converts written text to a phonemic representation, then converts the phonemic representation to waveforms that can be output as sound. TTS engines with different languages, dialects and specialized vocabularies are available through third-party publishers.<ref name="microsoft.com">{{cite web|url=http://support.microsoft.com/kb/306902 |title=How to configure and use Text-to-Speech in Windows XP and in Windows Vista |publisher=Microsoft  |date=2007-05-07 |access-date=2010-02-17}}</ref>

=== Android ===

Version 1.6 of [[Android (operating system)|Android]] added support for speech synthesis (TTS).<ref>{{cite web |author = Jean-Michel Trivi |date=2009-09-23 |url=http://android-developers.blogspot.com/2009/09/introduction-to-text-to-speech-in.html |title=An introduction to Text-To-Speech in Android |publisher=Android-developers.blogspot.com |access-date=2010-02-17}}</ref>

=== Internet ===
Currently, there are a number of [[application software|applications]], [[Plug-in (computing)|plugins]] and gadgets that can read messages directly from an [[e-mail client]] and web pages from a [[web browser]] or [[Google Toolbar]]. Some specialized software can narrate [[RSS|RSS-feeds]]. On one hand, online RSS-narrators simplify information delivery by allowing users to listen to their favourite news sources and to convert them to [[podcast]]s. On the other hand, on-line RSS-readers are available on almost any personal computer connected to the Internet. Users can download generated audio files to portable devices, e.g. with a help of [[podcast]] receiver, and listen to them while walking, jogging or commuting to work.

A growing field in Internet based TTS is web-based [[assistive technology]], e.g. 'Browsealoud' from a UK company and [[Readspeaker]]. It can deliver TTS functionality to anyone (for reasons of accessibility, convenience, entertainment or information) with access to a web browser. The non-profit project [[Wikipedia:WikiProject Spoken Wikipedia/Pediaphon|Pediaphon]] was created in 2006 to provide a similar web-based TTS interface to the Wikipedia.<ref>Andreas Bischoff, [http://www.dr-bischoff.de/research/pdf/bischoff_pediaphon_uwsi2007_final.pdf The Pediaphon – Speech Interface to the free Wikipedia Encyclopedia for Mobile Phones], PDA's and MP3-Players, Proceedings of the 18th International Conference on Database and Expert Systems Applications,  Pages: 575–579 {{ISBN|0-7695-2932-1}}, 2007</ref>

Other work is being done in the context of the [[W3C]] through the W3C Audio Incubator Group with the involvement of The BBC and Google Inc.

===Open source===
Some [[open-source software]] systems are available, such as:
* [[eSpeak]] which supports a broad range of languages.
* [[Festival Speech Synthesis System]] which uses diphone-based synthesis, as well as more modern and contemporary sounding techniques.
* [[gnuspeech]] which uses articulatory synthesis<ref>{{cite web|url=https://www.gnu.org/software/gnuspeech/ |title=gnuspeech |publisher=Gnu.org |access-date=2010-02-17}}</ref> from the [[Free Software Foundation]].

=== Others ===
* Following the commercial failure of the hardware-based Intellivoice, gaming developers sparingly used software synthesis in later games{{Citation needed|date=April 2020}}. Earlier systems from Atari, such as the [[Atari 5200]] (''Baseball'') and the [[Atari 2600]] (''[[Quadrun]]'' and ''Open Sesame''), also had games utilizing software synthesis.{{Citation needed|date=April 2020}}
* Some [[e-book readers]], such as the [[Amazon Kindle]], [[Samsung]] E6, [[PocketBook eReader]] Pro, [[enTourage eDGe]], and the Bebook Neo.
* The [[BBC Micro]] incorporated the Texas Instruments TMS5220 speech synthesis chip.
* Some models of Texas Instruments home computers produced in 1979 and 1981 ([[TI-99/4A|Texas Instruments TI-99/4 and TI-99/4A]]) were capable of text-to-phoneme synthesis or reciting complete words and phrases (text-to-dictionary), using a very popular Speech Synthesizer peripheral. TI used a proprietary [[codec]] to embed complete spoken phrases into applications, primarily video games.<ref>{{cite web |url=http://www.mindspring.com/~ssshp/ssshp_cd/ss_home.htm |title=Smithsonian Speech Synthesis History Project (SSSHP) 1986–2002 |publisher=Mindspring.com |access-date=2010-02-17 |archive-url=https://web.archive.org/web/20131003104852/http://amhistory.si.edu/archives/speechsynthesis/ss_home.htm |archive-date=2013-10-03 |url-status=dead }}</ref>
* [[IBM]]'s [[OS/2 Warp|OS/2 Warp 4]] included VoiceType, a precursor to [[IBM ViaVoice]].
* [[Global Positioning System|GPS]] Navigation units produced by [[Garmin]], [[Magellan Navigation|Magellan]], [[TomTom]] and others use speech synthesis for automobile navigation.
* [[Yamaha Corporation|Yamaha]] produced a music synthesizer in 1999, the [[Yamaha FS1R]] which included a Formant synthesis capability. Sequences of up to 512 individual vowel and consonant formants could be stored and replayed, allowing short vocal phrases to be synthesized.

=== Digital sound-alikes ===
At the 2018 [[Conference on Neural Information Processing Systems]] (NeurIPS) researchers from [[Google]] presented the work 'Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis', which [[Transfer learning|transfers learning]] from [[speaker recognition|speaker verification]] to achieve text-to-speech synthesis, that can be made to sound almost like anybody from a speech sample of only 5 seconds.<ref name="GoogleLearningTransferToTTS2018">

{{Citation
 | last1 = Jia
 | first1 = Ye
 | last2 = Zhang
 | first2 = Yu
 | last3 = Weiss
 | first3 = Ron J.
 | title = Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
 | journal = [[Advances in Neural Information Processing Systems]]
 | volume = 31
 | pages = 4485–4495
 | date = 2018-06-12
 | language = en
 | arxiv = 1806.04558
 }}

</ref>

Also researchers from [[Baidu Research]] presented a [[voice cloning]] system with similar aims at the 2018 NeurIPS conference,<ref name="Baidu2018">

{{Citation
 | last1 =  Arık
 | first1 = Sercan Ö.
 | last2 = Chen
 | first2 = Jitong
 | last3 = Peng
 | first3 = Kainan
 | last4 = Ping
 | first4 = Wei
 | last5 = Zhou
 | first5 = Yanqi
 | title = Neural Voice Cloning with a Few Samples
 | journal = [[Advances in Neural Information Processing Systems]]
 | volume = 31
 | year =2018
 | url = http://papers.nips.cc/paper/8206-neural-voice-cloning-with-a-few-samples
 | arxiv = 1802.06006
 }}

</ref> though the result is rather unconvincing.

By 2019 the digital sound-alikes found their way to the hands of criminals as [[NortonLifeLock|Symantec]] researchers know of 3 cases where digital sound-alikes technology has been used for crime.<ref name="BBC2019">
{{cite web
 |url= https://www.bbc.com/news/technology-48908736
 |title= Fake voices 'help cyber-crooks steal cash'
 |date= 2019-07-08
 |website= [[bbc.com]]
 |publisher= [[BBC]]
 |access-date= 2019-09-11
 }}
</ref><ref name="WaPo2019">
{{cite news
 |url= https://www.washingtonpost.com/technology/2019/09/04/an-artificial-intelligence-first-voice-mimicking-software-reportedly-used-major-theft/
 |title= An artificial-intelligence first: Voice-mimicking software reportedly used in a major theft
 |last= Drew
 |first= Harwell
 |date= 2019-09-04
 |newspaper= Washington Post
 |access-date= 2019-09-08
 }}
</ref>

This increases the stress on the disinformation situation coupled with the facts that 
* [[Human image synthesis]] since the early 2000s has improved beyond the point of human's inability to tell a real human imaged with a real camera from a simulation of a human imaged with a simulation of a camera.
*  2D video forgery techniques were presented in 2016 that allow [[Real-time computing#Near real-time|near real-time]] counterfeiting of [[facial expressions]] in existing 2D video.<ref name="Thi2016">{{cite web
  | last = Thies
  | first = Justus
  | title = Face2Face: Real-time Face Capture and Reenactment of RGB Videos
  | publisher = Proc. Computer Vision and Pattern Recognition (CVPR), IEEE
  | year = 2016
  | url = http://www.graphics.stanford.edu/~niessner/thies2016face.html
  | access-date =  2016-06-18}}
</ref>
* In [[SIGGRAPH]] 2017 an audio driven digital look-alike of upper torso of Barack Obama was presented by researchers from [[University of Washington]]. It was driven only by a voice track as source data for the animation after the training phase to acquire [[lip sync]] and wider facial information from training material consisting of 2D videos with audio had been completed.<ref name="Suw2017">{{Citation
 | last1 = Suwajanakorn | first1 = Supasorn 
 | last2 = Seitz | first2 = Steven 
 | last3 = Kemelmacher-Shlizerman | first3 = Ira 
 | title = Synthesizing Obama: Learning Lip Sync from Audio
 | publisher = [[University of Washington]]
 | year = 2017 
 | url = http://grail.cs.washington.edu/projects/AudioToObama/
 | access-date = 2018-03-02 }}
</ref>
In March 2020, a [[freeware]] web application called 15.ai that generates high-quality voices from an assortment of fictional characters from a variety of media sources was released.<ref name="Batch042020">
{{cite web|last=Ng|first=Andrew|date=2020-04-01|title=Voice Cloning for the Masses|url=https://blog.deeplearning.ai/blog/the-batch-ai-against-coronavirus-datasets-voice-cloning-for-the-masses-finding-unexploded-bombs-seeing-see-through-objects-optimizing-training-parameters|url-status=dead|archive-url=https://web.archive.org/web/20200807111844/https://blog.deeplearning.ai/blog/the-batch-ai-against-coronavirus-datasets-voice-cloning-for-the-masses-finding-unexploded-bombs-seeing-see-through-objects-optimizing-training-parameters|archive-date=2020-08-07|access-date=2020-04-02|website=deeplearning.ai|publisher=The Batch}}
</ref> Initial characters included [[GLaDOS]] from ''[[Portal (series)|Portal]]'', [[Twilight Sparkle]] and [[Fluttershy]] from the show ''[[My Little Pony: Friendship Is Magic]]'', and the [[Tenth Doctor]] from ''[[Doctor Who]]''.

== Speech synthesis markup languages ==

A number of [[markup language]]s have been established for the rendition of text as speech in an [[XML]]-compliant format. The most recent is [[Speech Synthesis Markup Language]] (SSML), which became a [[W3C recommendation]] in 2004. Older speech synthesis markup languages include Java Speech Markup Language ([[JSML]]) and [[SABLE]]. Although each of these was proposed as a standard, none of them have been widely adopted.{{citation needed|date=January 2022}}

Speech synthesis markup languages are distinguished from dialogue markup languages. [[VoiceXML]], for example, includes tags related to speech recognition, dialogue management and touchtone dialing, in addition to text-to-speech markup.{{citation needed|date=January 2022}}

== Applications ==

Speech synthesis has long been a vital assistive technology tool and its application in this area is significant and widespread. It allows environmental barriers to be removed for people with a wide range of disabilities. The longest application has been in the use of [[screen reader]]s for people with visual impairment, but text-to-speech systems are now commonly used by people with [[dyslexia]] and other [[Reading disability|reading disabilities]] as well as by pre-literate children.<ref>{{Cite journal |last1=Brunow |first1=David A. |last2=Cullen |first2=Theresa A. |date=2021-07-03 |title=Effect of Text-to-Speech and Human Reader on Listening Comprehension for Students with Learning Disabilities |url=https://www.tandfonline.com/doi/full/10.1080/07380569.2021.1953362 |journal=Computers in the Schools |language=en |volume=38 |issue=3 |pages=214–231 |doi=10.1080/07380569.2021.1953362 |hdl=11244/316759 |s2cid=243101945 |issn=0738-0569|hdl-access=free }}</ref> They are also frequently employed to aid those with severe [[speech impairment]] usually through a dedicated [[voice output communication aid]].<ref>{{Cite book |last1=Triandafilidi |first1=Ioanis I. |last2=Tatarnikova |first2=T. M. |last3=Poponin |first3=A. S. |title=2022 Wave Electronics and its Application in Information and Telecommunication Systems (WECONF) |chapter=Speech Synthesis System for People with Disabilities |date=2022-05-30 |chapter-url=https://ieeexplore.ieee.org/document/9803600 |location=St. Petersburg, Russian Federation |publisher=IEEE |pages=1–5 |doi=10.1109/WECONF55058.2022.9803600 |isbn=978-1-6654-7083-4|s2cid=250118756 }}</ref> Work to personalize a synthetic voice to better match a person's personality or historical voice is becoming available.<ref>{{Cite book |last1=Zhao |first1=Yunxin |last2=Song |first2=Minguang |last3=Yue |first3=Yanghao |last4=Kuruvilla-Dugdale |first4=Mili |title=2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI) |chapter=Personalizing TTS Voices for Progressive Dysarthria |date=2021-07-27 |chapter-url=https://ieeexplore.ieee.org/document/9508522 |location=Athens, Greece |publisher=IEEE |pages=1–4 |doi=10.1109/BHI50953.2021.9508522 |isbn=978-1-6654-0358-0|s2cid=236982893 }}</ref>  A noted application, of speech synthesis, was the [[Reading machine|Kurzweil Reading Machine for the Blind]] which incorporated text-to-phonetics software based on work from [[Haskins Laboratories]] and a black-box synthesizer built by [[Votrax]].<ref>{{Cite journal |date=1984 |title=Evolution of Reading Machines for the Blind: Haskins Laboratories" Research as a Case History |url=https://www.rehab.research.va.gov/jour/84/21/1/pdf/cooper.pdf |journal=[[Journal of Rehabilitation Research and Development]] |volume=21 |issue=1}}</ref>
[[File:Stephen Hawking.StarChild.jpg|thumb|upright=.7|left|[[Stephen Hawking]] was one of the most famous people to use a speech computer to communicate.]]
Speech synthesis techniques are also used in entertainment productions such as games and animations. In 2007, Animo Limited announced the development of a software application package based on its speech synthesis software FineSpeech, explicitly geared towards customers in the entertainment industries, able to generate narration and lines of dialogue according to user specifications.<ref>{{cite news|url=http://www.animenewsnetwork.com/news/2007-05-02/speech-synthesis-software |title=Speech Synthesis Software for Anime Announced |work=Anime News Network |date=2007-05-02 |access-date=2010-02-17}}</ref> The application reached maturity in 2008, when NEC [[Biglobe]] announced a web service that allows users to create phrases from the voices of characters from the Japanese [[anime]] series ''[[Code Geass: Lelouch of the Rebellion R2]]''.<ref>{{cite web|url=http://www.animenewsnetwork.com/news/2008-09-09/code-geass-voice-synthesis-service-offered-in-japan |title=Code Geass Speech Synthesizer Service Offered in Japan |publisher=Animenewsnetwork.com |date=2008-09-09 |access-date=2010-02-17}}</ref> 15.ai has been frequently used for [[content creation]] in various [[fandom]]s, including the [[My Little Pony: Friendship Is Magic fandom|''My Little Pony: Friendship Is Magic'' fandom]], the ''[[Team Fortress 2]]'' fandom, the ''[[Portal (series)|Portal]]'' fandom, and the ''[[SpongeBob SquarePants]]'' fandom.{{citation needed|date=June 2024}}

Text-to-speech for disability and impaired communication aids have become widely available. Text-to-speech is also finding new applications; for example, speech synthesis combined with [[speech recognition]] allows for interaction with mobile devices via [[natural language processing]] interfaces. Some users have also created AI [[virtual assistant]]s using 15.ai and external voice control software.<ref name="automaton2"/><ref name="Denfaminicogamer2"/>

Text-to-speech is also used in second language acquisition. Voki, for instance, is an educational tool created by Oddcast that allows users to create their own talking avatar, using different accents. They can be emailed, embedded on websites or shared on social media.

Content creators have used voice cloning tools to recreate their voices for podcasts,<ref name=":162">{{Cite web |date=2023-06-20 |title=Now hear this: Voice cloning AI startup ElevenLabs nabs $19M from a16z and other heavy hitters |url=https://venturebeat.com/ai/now-hear-this-voice-cloning-ai-startup-elevenlabs-nabs-19m-from-a16z-and-other-heavy-hitters/ |access-date=2023-07-25 |website=VentureBeat |language=en-US}}</ref><ref>{{Cite web |date=April 9, 2023 |title=Sztuczna inteligencja czyta głosem Jarosława Kuźniara. Rewolucja w radiu i podcastach |url=https://www.press.pl/tresc/75988,sztuczna-inteligencja-czyta-glosem-jaroslawa-kuzniara_-to-zapowiedz-rewolucji-w-radiu-i-podcastach |access-date=2023-04-25 |website=Press.pl |language=pl}}</ref> narration,<ref name=":13"/> and comedy shows.<ref>{{Cite magazine |last=Knibbs |first=Kate |title=Generative AI Podcasts Are Here. Prepare to Be Bored |url=https://www.wired.com/story/generative-ai-podcasts-boring/ |magazine=Wired |language=en-US |issn=1059-1028 |access-date=2023-07-25}}</ref><ref>{{Cite web |last=Suciu |first=Peter |title=Arrested Succession Parody On YouTube Features 'Narration' By AI-Generated Ron Howard |url=https://www.forbes.com/sites/petersuciu/2023/05/09/arrested-succession-parody-on-youtube-features-narration-by-ai-generated-ron-howard/ |access-date=2023-07-25 |website=Forbes |language=en}}</ref><ref>{{Cite news |last=Fadulu |first=Lola |date=2023-07-06 |title=Can A.I. Be Funny? This Troupe Thinks So. |language=en-US |work=The New York Times |url=https://www.nytimes.com/2023/07/06/nyregion/artificial-intelligence-comedy.html |access-date=2023-07-25 |issn=0362-4331}}</ref> Publishers and authors have also used such software to narrate audiobooks and newsletters.<ref name=":2">{{Cite web |last=Kanetkar |first=Riddhi |title=Hot AI startup ElevenLabs, founded by ex-Google and Palantir staff, is set to raise $18 million at a $100 million valuation. Check out the 14-slide pitch deck it used for its $2 million pre-seed. |url=https://www.businessinsider.com/elevenlabs-ai-voice-intelligence-startup-raises-2-million-2023-1 |access-date=2023-07-25 |website=Business Insider |language=en-US}}</ref><ref name=":02">{{Cite web |date=January 30, 2023 |title=AI-Generated Voice Firm Clamps Down After 4chan Makes Celebrity Voices for Abuse |url=https://www.vice.com/en/article/ai-voice-firm-4chan-celebrity-voices-emma-watson-joe-rogan-elevenlabs/ |access-date=2023-02-03 |website=Vice.com |language=en}}</ref> Another area of application is AI video creation with talking heads. Webapps and video editors like Elai.io or [[Synthesia (company)|Synthesia]] allow users to create video content involving AI avatars, who are made to speak using text-to-speech technology.<ref>{{cite web |title=Usage of text-to-speech in AI video generation |url=https://elai.io/ |website=elai.io |access-date=10 August 2022}}</ref><ref>{{cite web |title=AI Text to speech for videos|url=https://www.synthesia.io/text-to-speech|website=synthesia.io |access-date=12 October 2023}}</ref>

Speech synthesis is a valuable computational aid for the analysis and assessment of speech disorders. A [[voice quality]] synthesizer, developed by Jorge C. Lucero et al. at the [[University of Brasília]], simulates the physics of [[phonation]] and includes models of vocal frequency jitter and tremor, airflow noise and laryngeal asymmetries.<ref name=":0" /> The synthesizer has been used to mimic the [[timbre]] of [[dysphonic]] speakers with controlled levels of roughness, breathiness and strain.<ref name=":1" />

=== Singing synthesis ===
{{excerpt|Music technology (electronic and digital)|Vocal synthesis after 2010s}}

== See also ==
{{columns-list|colwidth=22em|
* [[Chinese speech synthesis]]
* [[Comparison of speech synthesizers]]
* [[List of screen readers]]
* [[Orca (assistive technology)]]
* [[Paperless office]]
* [[Silent speech interface]]
* [[Speech-generating device]]
* [[Speech processing]]
* [[Text to speech in digital television]]
}}

== References ==
{{Reflist}}

== External links ==
{{Commons category}}
*[https://www.youtube.com/watch?v=CE6zy8aUwtQ Simulated singing with the singing robot Pavarobotti] or a description from the [[BBC]] on [https://www.youtube.com/watch?v=SNqNM6Ccck8 how the robot synthesized the singing].
{{sound synthesis types}}
{{Speech synthesis}}
{{Natural Language Processing}}

{{Authority control}}

{{DEFAULTSORT:Speech Synthesis}}
[[Category:Speech synthesis| ]]
[[Category:Applications of artificial intelligence]]
[[Category:Assistive technology]]
[[Category:Auditory displays]]
[[Category:Computational linguistics]]
[[Category:History of human–computer interaction]]