Teaching Small Language Models Their Own Reasoning Style Boosts Problem-Solving

Researchers have discovered that smaller language models learn to reason more effectively when trained on solutions expressed in language patterns they can naturally produce, rather than complex reasoning traces from much larger models.

The core challenge#

Training small models to think through problems carefully faces a fundamental mismatch. When researchers use supervised fine-tuning (SFT) with reasoning chains from larger, more capable models, those traces often contain rare word choices or reasoning patterns the small model struggles to generate on its own. Conversely, purely on-policy learning (where models learn only from their own attempts) provides weak training signals, since small models rarely produce correct solutions without guidance.

A new balancing act#

The team introduced Interleaved-Policy Distillation (IPD), which generates training data by letting the student model itself propose modifications to teacher-generated solutions. When a larger model's reasoning trace includes tokens the small model finds unlikely, the student can suggest an alternative from its own vocabulary. The teacher then continues reasoning from that student-compatible point. This creates reasoning solutions that remain strong while staying within the small model's learned language patterns.

Real-world gains#

The method improved performance across multiple architectures and datasets. On seven standard reasoning benchmarks, Qwen3-0.6B performance rose from 24.55 to 29.72 when using IPD. The approach outperformed simpler alternatives—such as just filtering incompatible traces or masking loss weights—and expanded the range of problems small models could tackle.

Why it matters#

The finding reframes a core challenge in efficient AI: that more powerful guidance doesn't automatically translate to faster learning. Small models reason better when they acquire strong solutions in a form they can themselves produce, suggesting that training quality depends not just on the content of solutions but on their accessibility to the learner.