OpenAI's o1 Model Shows AGI-Like Reasoning Across Math, Code, and Science

Researchers have conducted a sweeping assessment of OpenAI's o1-preview model, testing its reasoning capabilities across eight major subject areas and finding performance that frequently matches or exceeds human expertise in demanding tasks.

Strong Performance Across Domains#

The study tested o1-preview on a range of cognitively intensive challenges. The model achieved an 83.3% success rate on competitive programming problems—a benchmark where most AI systems struggle—and demonstrated flawless accuracy on high school mathematics, delivering step-by-step reasoning for each solution. It also generated more accurate radiology reports than comparison models and showed sophisticated understanding when analyzing specialized medical cases, from legal documents to anthropological and geological reasoning.

Key Capabilities#

o1-preview demonstrated particular strength in several areas:

• Competitive coding: Outperformed many human experts by solving complex algorithmic problems at high frequency
• Medical imaging: Produced superior radiology report generation compared to other evaluated models
• Mathematics: Achieved perfect scores on high school-level reasoning with detailed working
• Chip design: Exceeded specialized models in electronic design automation script creation and bug analysis
• Finance: Displayed comprehensive knowledge for quantitative investing and statistical modeling
• Multi-domain reasoning: Integrated knowledge across fields to tackle intricate problems requiring cross-disciplinary understanding

What Remains Challenging#

Despite these advances, some limitations emerged. The model occasionally stumbled on simpler problems and encountered difficulty with certain highly specialized or niche concepts, suggesting reasoning capability does not universally scale down to easier tasks or translate uniformly to all expert domains.

Implications for AGI#

The breadth and depth of o1-preview's performance signal meaningful progress toward artificial general intelligence, though the uneven performance profile—excelling at complex synthesis while occasionally faltering on basic tasks—underscores that current models remain far from truly general reasoning.