Generative Outcome Verification Outperforms Step-by-Step Scoring for AI Reasoning Across Domains

Researchers have challenged a widely held assumption in AI safety: that fine-grained supervision of reasoning steps always outperforms simpler final-answer verification. A new paper on arXiv evaluates how different reward model architectures perform when validating large language model outputs across 14 distinct domains.

Four Approaches Tested at Scale#

The study compared four reward model variants designed to distinguish correct reasoning from flawed logic during test-time scaling:

• Discriminative outcome models (dORM): judge only the final answer
• Discriminative process models (dPRM): score each intermediate reasoning step
• Generative outcome models (gORM): generate and verify the final result
• Generative process models (gPRM): generate assessments of each step

Counterintuitive Findings on Step-Level Scoring#

The results subvert conventional wisdom in several ways. Discriminative outcome and process models performed comparably—contradicting the expectation that step-by-step signals would yield substantial improvements. More surprisingly, generative process models underperformed their peers, despite the intuitive appeal of evaluating reasoning granularly.

The root cause, the authors argue, traces back to label noise accumulation. When automatic labeling from LLMs is applied to each step of a reasoning chain, errors compound during aggregation. Long reasoning trajectories—especially those involving self-correction—become particularly vulnerable to these cascading mistakes.

Generative Outcome Models Win Across the Board#

Generative outcome verification emerged as the most robust approach, delivering consistent gains across every tested domain. This simpler method, which generates and then verifies final answers without explicit step-level supervision, proved surprisingly effective at handling the complexity of real-world reasoning tasks.

Theoretical analysis in the paper shows that stepwise error accumulation intensifies as reasoning chains grow longer, explaining why process-level models struggled in practice. The findings suggest that the right target—outcome-level verification—matters more than fine-grained intermediate signals in multi-domain deployment scenarios.