OpenAI's o3-mini outperforms o1-mini through sharper reasoning, not longer chains

OpenAI's newer reasoning model o3-mini achieves higher accuracy than its predecessor o1-mini on mathematical problems without requiring proportionally longer reasoning chains, according to a technical analysis published on arXiv. This finding challenges assumptions about how test-time compute translates into performance gains in large language models.

What the research shows#

Researchers conducted a systematic comparison of reasoning token usage and accuracy across o1-mini and o3-mini variants on the Omni-MATH benchmark. Their key discovery: o3-mini delivers superior performance through more efficient reasoning rather than simply generating more tokens. Additionally, the team observed a counterintuitive trend—across all tested models, longer reasoning chains are actually associated with lower accuracy, suggesting that extra computation alone doesn't guarantee better results.

How models differ in scaling behavior#

The improvement appears generational rather than proportional. When comparing the more capable o3-mini (high) variant to o3-mini (medium), the high variant achieves only marginal accuracy gains but allocates substantially more reasoning tokens across all problems—even those the medium version already handles correctly. The accuracy decline from longer chains, however, proves significantly smaller in more advanced models, indicating that newer generations deploy test-time compute with greater sophistication.

Implications for AI efficiency and evaluation#

These findings suggest that future progress in reasoning models will depend less on raw computational scaling and more on architectural and training refinements that improve reasoning quality. The results carry practical implications for efficiency in deployment, methods for scaling reasoning systems, and how researchers should evaluate model improvements in mathematical reasoning tasks.