Watermarking AI Outputs Fails Across Most World Languages, New Study Shows

Despite their billing as multilingual, existing approaches to watermark large language model outputs collapse when tested against translation attacks in non-English and other widely-used languages, according to fresh research from computer scientists.

The problem with current watermarking#

Large language models now generate text across the globe, and watermarking techniques aim to make their outputs detectable and traceable in any language. Yet when researchers put these methods to the test on medium- and low-resource tongues, the watermarks faded away. The gap exposes a fundamental flaw: most existing systems have only been validated on a handful of high-resource languages like English, German, or French—not the majority of the world's linguistic diversity.

The root cause traces back to how modern language models tokenize text. When a model's vocabulary contains too few complete-word tokens for a particular language, the semantic clustering that watermarking relies on breaks down, rendering the watermark undetectable after translation.

A new detection method across 126 languages#

To address this limitation, researchers introduced STEAM, a detection approach that searches across 126 candidate languages using Bayesian optimization to find the back-translation that most effectively recovers the watermark's strength. The method works as a layer on top of any existing watermarking technique, making it agnostic to the underlying approach.

Key features of STEAM include:

• Language-agnostic: Compatible with different watermarking methods and tokenizers
• Non-invasive: Requires no modification to the original watermarking scheme
• Scalable: Easily extensible to new languages beyond the initial 126 tested
• Multilingual-first: Designed to level the playing field across linguistic communities

Performance gains and implications#

Testing showed meaningful improvements: an average gain of +0.23 AUC (area under the curve) and a true positive rate increase of +37 percentage points at a 1% false positive threshold. These results suggest that back-translation, rather than direct detection in the target language, unlocks watermark recovery across linguistic borders.