Selective data quality beats volume in literary machine translation training

Literary translation has always offered multiple valid interpretations of the same text—a single poem or novel can be rendered into another language in fundamentally different yet equally legitimate ways. A new study from arXiv explores how machine translation systems can harness this diversity to improve their performance.

The multi-reference advantage#

Researchers investigated how to best use datasets containing multiple human translations of literary works. Rather than treating all reference translations equally, they developed a filtering method based on semantic similarity—measuring how much variation exists between different translations while ensuring each remains faithful to the source material.

The findings suggest a counterintuitive insight: more data isn't always better. Fine-tuning translation models on medium to high semantic similarity pairs significantly outperformed models trained on low semantic similarity data. Remarkably, using only the filtered high-quality subset achieved results comparable to or better than using the entire unfiltered dataset.

Human expertise vs. synthetic alternatives#

The research also examined whether large language models could substitute for human expert translators. While synthetic translations generated by LLMs offer practical convenience and cost efficiency, the results were conclusive: fine-tuning on human expert translations consistently exceeded performance from synthetically augmented data across both automatic metrics and human evaluations.

This finding underscores a persistent gap between AI-generated and expert human work in the specialized domain of literary translation, where nuance, cultural context, and stylistic choices matter enormously.