New technique balances pronoun accuracy with overall translation quality

Machine translation systems struggle with context-dependent decisions like pronoun reference, which require understanding discourse beyond a single sentence. A new study presents ProNMT, an iterative self-training approach that tackles this challenge by blending broad translation quality signals with fine-grained linguistic feedback.

The core challenge#

Pronouns are notoriously difficult for machine translation because their correct choice depends on what comes before—context that extends beyond the immediate sentence being translated. While context-aware models can theoretically access this information, standard training methods don't explicitly push the system to get pronouns right, treating them as just another part of the overall translation task.

How ProNMT works#

The method operates through a cycle of sampling and refinement. For each sentence and its preceding source material, ProNMT generates multiple candidate translations. These candidates are then scored using two signals: a reference-free quality estimation metric that judges overall translation goodness, and a pronoun-specific label derived from reference translations. The system then fine-tunes on whichever candidate scores highest across both dimensions.

Testing on filtered English–German Europarl data and English–French News Commentary datasets, ProNMT outperformed baseline context-aware supervised fine-tuning on both BLEU and COMET metrics—standard benchmarks for translation quality.

Why balance matters#

Ablation studies revealed a critical insight: feedback focused only on pronouns can actually harm broader translation quality. Similarly, hard binary right-or-wrong signals underperformed when compared to confidence-weighted feedback that captures degrees of correctness. The researchers concluded that targeted linguistic feedback is most effective when paired with global quality signals and the contextual information needed to make the targeted decision.

The team has released their code publicly to enable further research in this direction.