Two critical gaps limit agentic software engineering—and testing can't fully bridge them

The traditional development cycle relies on agents or humans iterating through code revisions until test suites approve the result. But passing formal verification against a specification tells us less than we assume: requirements don't fully capture what stakeholders actually need, and test environments don't perfectly mirror real-world conditions.

A fresh analysis reframes the core challenge in AI-assisted software engineering around two structural limitations, labeled the requirement gap and model gap. These explain why even formally correct implementations can fail once deployed, and why verification-focused approaches have inherent blindspots.

The Two-Gap Problem#

Requirements are approximations of human intent. Test models approximate the actual deployment environment. Neither can account for every nuance or edge case in an open, evolving world. When software engineers encounter reward hacking or hallucination errors—two common failure modes for agentic systems—the root cause often traces back to these gaps: agents exploit omissions in specifications, or fabricate assumptions that don't hold in reality.

The insight is that neither gap can truly be "closed" in systems that operate in the real world. Instead, the goal must shift from eliminating these gaps to progressively narrowing them over time.

A Revised Development Loop#

Rather than stopping after predeployment verification, the researchers propose an assurance-revision loop that treats deployed behavior as the ground truth. When users or stakeholders reject the system's output, that feedback should trigger revisions to the requirements, the test environment model, or the evaluator itself.

This reframes assured agentic development as a resource-allocation problem: teams must balance three resources—human judgment, agent capability, and computational cost. The two bottlenecks directly mirror the two gaps. Closing the requirement gap depends on human oversight and interpretation. Closing the model gap depends on accurate, often expensive evaluation of real-world conditions.

Reality as Final Judge#

Predeployment testing remains useful but imperfect. The ultimate verification is whether the deployed system behaves acceptably under actual conditions. This shifts the burden from static correctness proofs to continuous, evidence-based refinement through real-world deployment and feedback.