In plain words: They compared the same web pages across many languages and checked how good the translations were to spot machine-made ones. Machine-translated junk, often from low-quality English pages, fills most translations and a big share of all web content in smaller languages.
Abstract · A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism
We show that content on the web is often translated into many languages, and the low quality of these multi-way translations indicates they were likely created using Machine Translation (MT). Multi-way parallel, machine generated content not only dominates the translations in lower resource languages; it also constitutes a large fraction of the total web content in those languages. We also find evidence of a selection bias in the type of content which is translated into many languages, consistent with low quality English content being translated en masse into many lower resource languages, via MT. Our work raises serious concerns about training models such as multilingual large language models on both monolingual and bilingual data scraped from the web.
Brian Thompson, Mehak Preet Dhaliwal, Peter Frisch, Tobias Domhan, Marcello Federico
arXiv:2401.05749 · cs.CL, cs.AI · submitted Jan 11, 2024 · updated Jun 5, 2024
abstract · pdf · html · Accepted at ACL Findings 2024
I can read three languages, and I'd like my browser to pick the "main" version of a page if it happens to be in any of those three languages, and only resort to translated ones if necessary. But there is no way to get this behaviour!
And I'd like so much for google to serve me Wikipedia pages in Italian or French if the subject concerns Italy or France, and only resort to English as a fallback, but no way, again!
If I set my main language to something other than English, I'll always get poorly translated MSDN documentation, and ugly web surfing in general. As such, I'm stuck with poorly translated youtube video titles, and English Wikipedia when searching about a monument in my hometown.