New approach embeds reasoning directly into language model training rather than adding it afterward

A team of researchers has unveiled Ouro, a family of language models that fundamentally rethink where and how reasoning happens during AI training. Rather than relying on chain-of-thought prompting—where models generate their working out as visible text after training—these Looped Language Models embed reasoning directly into the pre-training process itself.

How latent reasoning works#

The key innovation lies in iterative computation happening inside the model's hidden layers rather than in its output text. During training, the models learn to process information through multiple internal loops, refining their understanding of a problem without generating visible intermediate steps. This approach is guided by an entropy-regularized objective that lets each part of the model decide how many internal iterations it needs—creating an adaptive system where some computations require more depth than others.

The models were trained on 7.7 trillion tokens, yielding two main versions: a 1.4 billion parameter model and a 2.6 billion parameter model, both of which showed surprising performance gains.

Results that challenge conventional scaling#

In benchmark testing across a range of tasks, these smaller Looped Language Models matched or exceeded the performance of substantially larger baseline systems—including models with up to 12 billion parameters. Controlled experiments revealed that this advantage stemmed not from larger knowledge bases but from superior ability to manipulate and reason with existing knowledge.

The reasoning traces produced by the latent approach also aligned more closely with the models' final answers than those generated by explicit chain-of-thought methods, suggesting the internal reasoning was more tightly integrated with decision-making.

Implications for AI scaling#

The findings suggest that iterative computation in hidden space represents a genuine alternative direction for improving reasoning in large language models. Rather than scaling model size or relying on visible step-by-step reasoning, this approach optimizes how information flows through layers during training. The work opens questions about whether future scaling efforts should balance increases in parameters with deeper internal reasoning mechanisms.

The researchers have open-sourced Ouro, making the models and methods available to the research community.