New Framework Trains Smaller Language Models to Rival GPT-4o and Gemini on Translation

A team of researchers has identified and solved a fundamental problem in how large language models learn to translate between languages: the systems being used to teach them often reward superficially smooth-sounding but inaccurate translations over faithful ones.

The issue stems from Quality Estimation (QE) models, which typically guide translation training by measuring how good a candidate translation is. The team found these models systematically overlook specific kinds of errors—partially hallucinated phrases and omitted content—that don't dramatically disrupt the surface fluency of the output.

The Core Problem#

Current reward signals used in machine translation training exhibit a persistent bias toward fluency over fidelity. A translation might flow naturally in English but miss key parts of the source material or introduce details that weren't there. Existing evaluation methods tend to give such outputs higher scores than translations that are accurate but less polished-sounding, creating a training signal that pushes models in the wrong direction.

This bias becomes especially problematic when aligning LLMs with human preferences, where both accuracy and readability matter equally.

Multi-Perspective Multi-Pair Optimization#

The researchers propose M²PO (Multi-Perspective Multi-Pair Preference Optimization), a data-focused approach with two main components:

• Dual-perspective mechanism: Decouples semantic fidelity from fluency and uses a curriculum learning strategy to prioritize faithfulness during training, directly countering the fluency bias.
• Multi-pair objective: Rather than the typical approach of comparing only the best and worst translation candidates, the framework leverages the entire list of candidates to capture subtle error patterns that fall between perfect and severely flawed translations.

Results and Implications#

Testing on multiple benchmarks—WMT23, WMT24, and FLORES-200—showed that M²PO enables a 9-billion-parameter model to match or exceed widely used open-source baselines and reach performance parity with proprietary systems including GPT-4o and Gemini-2.0-Flash.

The authors plan to release code and datasets accompanying the work, offering other researchers a path toward building efficient yet high-fidelity translation systems without relying on costly commercial APIs.