Editing Parallel text (section)

==Noise in corpora==

Large corpora used as training sets for [[machine translation]] algorithms are usually extracted from large bodies of similar sources, such as databases of news articles written in the first and second languages describing similar events.

However, extracted fragments may be noisy, with extra elements inserted in each corpus. Extraction techniques can differentiate between [[bilingual]] elements represented in both corpora and [[monolingual]] elements represented in only one corpus in order to extract cleaner parallel fragments of bilingual elements. Comparable corpora are used to directly obtain knowledge for translation purposes. High-quality parallel data is difficult to obtain, however, especially for under-resourced languages.<ref>{{Cite journal |last=Wołk |first=Krzysztof |date=2015 |title=Noisy-Parallel and Comparable Corpora Filtering Methodology for the Extraction of Bi-Lingual Equivalent Data at Sentence Level |journal=Computer Science |volume=16 |issue=2 |pages=169–184 |arxiv=1510.04500 |bibcode=2015arXiv151004500W |doi=10.7494/csci.2015.16.2.169 |doi-access=free |s2cid=12860633}}</ref>