New Two-Way Reward Model Improves How AI Systems Evaluate Their Own Reasoning

A research team has developed an improved approach to evaluating the quality of AI reasoning by scoring intermediate steps in two directions simultaneously—forward and backward—rather than just left-to-right as traditional methods do.

What the research proposes#

The new technique, called Bidirectional Process Reward Model (BiPRM), evaluates reasoning trajectories in large language models from multiple angles. While conventional Process Reward Models (PRMs) score each step in a solution only by looking at everything before it, BiPRM adds a parallel stream that reads the same steps in reverse order. A gating mechanism then intelligently combines the scores from both directions into a single overall quality assessment.

This bidirectional perspective allows the model to leverage fuller context—each evaluation pass can see information that the other cannot, leading to more balanced judgments about step correctness.

Performance and efficiency gains#

The efficiency of the approach stands out. Despite doubling the evaluation pathways, BiPRM adds just 0.3% more parameters to the underlying model, while the actual runtime cost is only about 5% slower due to parallel execution of both streams.

In testing across multiple benchmarks and AI model architectures, BiPRM delivered substantial improvements:

• 10.6% average relative gain in solution-level configurations (54 scenarios tested)
• 37.7% improvement in step-level error detection tasks (12 scenarios)

Why this matters#

Process reward models have become a key technique for steering large language models toward stronger reasoning. By moving beyond unidirectional evaluation, BiPRM demonstrates that broader contextual awareness during scoring directly translates to better error detection and overall solution quality. The findings suggest bidirectional thinking may unlock meaningful advances in how AI systems validate their own problem-solving steps.