AI
Which Language Models Make the Best Multilingual Teachers? New Study Breaks the Bigger-Is-Better Myth
Researchers found that the largest models don't automatically produce the best synthetic training data for teaching smaller models multiple languages—what matters more are specific data qualities.
When organizations want to teach smaller language models to work across many tongues, they often turn to large models to generate synthetic training data. Yet picking which large model to use has typically been a guessing game, with teams defaulting to whatever the biggest option available is. A new study reveals this approach may waste resources and hurt results.
The Problem with Defaulting to Scale#
Researchers across multiple institutions systematically tested what actually makes an effective multilingual teacher model. Their key finding challenges conventional wisdom: simply using the largest model on the shelf doesn't guarantee good synthetic data or strong performance in downstream tasks.
The gap in capability matters especially in non-English languages, where even leading models often struggle. When teacher models produce low-quality synthetic data due to weak multilingual ability, student models trained on that data inherit and amplify those weaknesses.
How the Study Measured Success#
The team evaluated 10 different language models across 6 typologically diverse languages, generating more than 1.4 million supervised finetuning (SFT) examples and training 240 student models to measure real-world impact. They created a metric called Polyglot Score that connects intrinsic data quality measures to actual student model performance.
Results showed that Gemma 3 27B and Aya Expanse 32B consistently outperformed other options as teachers across different student base model families—not necessarily the largest competitors, but well-rounded performers.
What Actually Predicts Good Teaching#
The researchers drilled deeper to identify which data characteristics drive effectiveness. Model size alone turned out to be a poor predictor; instead, specific properties of the synthetic data captured the picture:
- Prompt diversity — varied input examples
- Response length — appropriately detailed outputs
- Response fluency — natural, high-quality language
These three factors together explained over 93.3% of variance in intrinsic data quality and reliably predicted how well student models would perform.
