New benchmark reveals AI agents struggle with proactive problem-solving beyond simple tasks

As language model-based agents become more sophisticated, they're expected to move beyond responding to explicit commands—instead spotting problems before users ask and solving them independently. But evaluating whether they can actually do this remains difficult. A new research framework called PROBE (Proactive Resolution Of BottlEnecks) reveals that current AI systems, even the most capable ones, struggle significantly with this kind of autonomous reasoning.

The challenge of measuring proactivity#

Existing benchmarks for testing AI agents tend to be narrow in scope, confining their evaluation to isolated problems within a single context. This makes it hard to assess whether agents can synthesize information across multiple sources or reason effectively over longer stretches of time—skills that would be essential for genuine proactive problem-solving. Researchers set out to build a more comprehensive measurement approach.

How PROBE works#

The framework breaks proactivity down into three sequential capabilities. First, an agent must search for unspecified issues—recognizing potential problems that haven't been explicitly mentioned. Second, it must identify specific bottlenecks that are actually worth addressing. Third, it must execute appropriate resolutions to those bottlenecks. This pipeline approach allows researchers to pinpoint exactly where and why agents fail.

Current limitations exposed#

When applied to leading large language models and popular agentic frameworks, the results are sobering. The highest end-to-end performance achieved—by both GPT-5 and Claude Opus-4.1—is just 40%. This means even frontier models succeed less than half the time when tasked with finding and resolving unanticipated problems. The research also maps out which failure modes are most common across different models, revealing systematic weaknesses in how these systems approach autonomous action.

What's next#

The authors highlight that their findings expose both the current limitations of agentic systems and avenues for improvement. As AI agents become more central to real-world workflows, the gap between reactive assistance and genuinely proactive problem-solving represents a significant frontier for AI research.