Study Reveals How Late Requirements Derail AI Coding Agents

When users clarify what they actually want after an AI coding agent has already started building, the result is measurably more wasted effort. A new study from arXiv digs into how late requirement changes ripple through AI-assisted development, revealing patterns that echo long-standing problems in traditional software engineering—but happening far more rapidly.

The Challenge#

Coding agents face a classic dilemma: they must implement changes before users can fully articulate what they need. In traditional software projects, this mismatch between stated intent and actual requirements has long been known to cause delays and budget overruns. But those studies tracked impact at release-cycle timescales. For AI-driven development, the cycle is compressed—and the churn is compounded.

Existing research on coding agents hasn't captured the full picture. Benchmark studies artificially lock requirements in place before implementation begins, while observational work counts how often users push back without measuring the actual code damage those changes inflict.

What the Research Found#

Researchers examined 3,553 real-world sessions from SWE-chat, a platform where users collaborate with coding agents. They tracked requirement arrivals along multiple dimensions and, where possible, replayed repository history to link each new requirement to the code it invalidated—measured as lines the agent later deleted or rewrote.

The findings were stark: requirements that emerged after initial implementation triggered roughly twice as much code invalidation as comparable non-requirement edits. This pattern held up across multiple robustness checks, including variations in how much code was deleted overall and how user turns were counted.

Notably, the burden showed no decline as sessions progressed, suggesting users don't learn to articulate requirements more clearly over time. Different types of operations produced similar rework costs once multiplicity was factored in, though some confidence intervals remained wide.

What It Means#

The research also tested whether advance warning helped. A controlled experiment found that when users revealed requirements late, agents shifted their implementation work after the disclosure. However, giving agents a heads-up in advance showed no measurable protective effect—they still overwrote prior code at similar rates.

Together, these results establish late requirement emergence as a quantifiable source of wasted development effort in AI-coding workflows, bridging the gap between real-world friction and measurable harm.