Researchers develop method to combine simple reasoning steps into complex AI reasoning

Machine learning researchers have developed a technique that enables large language models to reuse basic reasoning skills to tackle more complex problems they've never encountered before—without requiring labeled training data for each new challenge.

The problem with standard training#

The typical way to teach LLMs to reason involves collecting chain-of-thought (CoT) traces—step-by-step explanations of how to solve problems—from human annotators or other sources. But gathering these annotated examples is expensive and impractical when you want models to handle new types of reasoning tasks.

Current approaches struggle when models must combine multiple simpler skills to solve a harder problem. A model trained on individual reasoning tasks often fails to generalize well when asked to compose those skills in novel ways.

A composable approach to reasoning#

The researchers found that the standard way of formatting reasoning traces doesn't naturally support composition. By making modest adjustments to how atomic reasoning steps are recorded, they enabled models to better transfer and combine these skills. The key insight: reformatting CoT data to be "composable"—ensuring atomic reasoning steps can be cleanly combined—leads to measurable improvements when tackling new composite tasks.

The team tested two strategies for combining these atomic reasoning modules:

• Multitask learning: Training a single model on multiple atomic tasks simultaneously
• Model merging: Combining separate models trained on individual tasks

Both approaches, paired with composable CoT formatting, showed better zero-shot performance on target compositional tasks than standard baselines.

Going further with fine-tuning#

The combined models could be improved further by bootstrapping on a small amount of actual compositional examples through rejection sampling fine-tuning (RFT)—a technique that keeps only successful solutions and discards failures.

Experiments on string manipulation tasks and natural language reasoning compositions demonstrated that models trained with composable CoT formatting outperform multitask learning and continued fine-tuning approaches when operating within the same training data budget. This suggests a path toward building reasoning systems that generalize more efficiently beyond their initial training distribution.