Reasoning Distillation vs. Scaling: Study Questions Whether Longer Training Data Pays Off

A recent study from arXiv challenges a common assumption in AI development: that having models learn by mimicking detailed reasoning traces from powerful teachers is always worth the extra computational cost.

The Trade-off#

Reasoning distillation has become standard practice for building capable small language models. The approach has a fundamental constraint: because reasoning outputs are 5 to 20 times longer than traditional instruction fine-tuning (IFT) responses, developers must choose between two paths with the same compute budget—train a smaller model with reasoning traces, or train a larger model with simpler instructions.

What the Experiments Found#

Researchers conducted a controlled comparison by having a single teacher model generate both types of outputs for the same prompts, isolating the format of supervision as the only variable. They trained student models across five different sizes (from 0.5 billion to 14 billion parameters) and tested performance on 18 benchmarks.

The results were striking: at equivalent computational cost, standard instruction fine-tuning performed as well or better than reasoning distillation in most scenarios. Reasoning data only reached competitive performance on open-ended tasks—and only when models reached 7 billion parameters or larger.

A More Efficient Middle Ground#

The good news for reasoning advocates: a hybrid approach proved viable. When researchers combined just 25 to 50 percent reasoning data with standard instruction fine-tuning in a sequential curriculum, models captured most of the accuracy gains that pure reasoning distillation offered—but at significantly lower computational cost.

The findings suggest that practitioners may want to reconsider where reasoning distillation delivers genuine value, rather than treating it as a universal best practice.