Solution Diversity Emerges as Key Predictor of LLM Problem-Solving Success

A new research paper from arXiv suggests that the breadth of solutions large language models can produce for a single problem may be a reliable indicator of their problem-solving capacity. The study introduces solution divergence—the degree to which an LLM generates varied approaches to the same task—as both a performance metric and a training signal.

The Finding#

Researchers observed a consistent positive correlation between solution divergence and success rates across multiple LLM architectures. When models generated more diverse attempts at solving a problem, they tended to perform better overall. This relationship held across three distinct problem domains tested in the work.

Application to Training Methods#

The team demonstrated that solution divergence can enhance both supervised fine-tuning (SFT)—where models learn from labeled examples—and reinforcement learning (RL) approaches that rely on task feedback. By incorporating divergence as a guiding principle, they achieved measurable improvements in success rates across their evaluation domains.

Why It Matters#

The finding suggests that encouraging models to explore multiple solution paths, rather than converging too quickly on a single approach, may be beneficial for learning and inference alike. This perspective offers a simpler, more direct way to think about LLM training compared to purely feedback-based or data-centric methods, potentially opening new avenues for making these systems more reliable at complex reasoning tasks.