AI Agent Successfully Rewrote 717k-Line Codebase Without Human Review or Testing

Researchers have published a detailed case study of an AI coding agent successfully executing a large-scale architectural refactoring on a 717,725-line TypeScript production system—a task assessed as practically infeasible through conventional incremental changes—without human code review and without a pre-existing test oracle to validate correctness.

The Task: Breaking a Core UI Invariant#

The codebase, spanning 3,648 files, contained a fundamental lifetime guarantee: that a UI panel must remain open for the duration of any AI request. The goal was to dismantle this invariant so that a streaming generation could survive panel closure and later reattach to the same live stream without loss or duplication. This kind of deep architectural constraint typically demands a complete rewrite rather than piecemeal refactoring.

Specification-First Validation Protocol#

Rather than writing code directly, the agent followed a structured validation cycle:

• Specification phase: The agent authored a formal specification, then ran 14 refinement cycles comparing that specification against the existing source code to ensure correctness before implementation began.
• Implementation & testing: Once specification approval was reached, the agent performed atomic changes with a compile and test feedback loop.
• Verification phase: Post-implementation, 17 additional audit cycles checked the final code against the frozen specification.
• Convergence: The process continued through 31 total audit passes, identifying and correcting 201 defects before any human ran the program. Success was declared after two consecutive verification passes with zero findings.

Scope and Outcomes#

The refactoring touched 189 files (including 31 new files) and included an extraction phase affecting 288 files total, with 34,770 insertions and 16,422 deletions across two commits. Over roughly thirty sessions following the initial three days, the software performed as specified with no reported bugs. Total elapsed time: three days; total cost: USD 2,430.

The researchers have published the complete formal specification and over 1,500 pages of raw session logs (in French) as public evidence, enabling external inspection of the process and verification by other language models for consistency checking.