AI Agents Refactor 300K-Line Codebase in Three Weeks—but Skeptics Debate What Success Means

CodeScene's latest case study documents an ambitious refactoring project in which AI agents overhauled the Street Fighter III: 3rd Strike source code—a 300,000-line C decompilation—over the course of three weeks, spending just $4,000 in token costs. The transformation touched 726 files, modified 252,055 lines of code, and pushed the codebase's Code Health score from 5.6 to a perfect 10.0.

Adam Tornhill, CodeScene's founder, called this the first instance of "superhuman AI performance at scale" he has witnessed in three decades of studying large systems. Yet the result has divided practitioners and researchers—not over whether it happened, but over what it actually demonstrates about AI's real-world capabilities.

How the Agents Worked#

The refactoring campaign relied on two key technical innovations. The first was a measurable objective: agents used the CodeHealth MCP Server, which assigned a deterministic score at each step, allowing them to know immediately whether a transformation improved or degraded the codebase. The second was a strict correctness mechanism—a replay-trace harness that compared the game's state frame-by-frame before and after each change, ensuring that the refactored code continued to behave identically.

The work produced 2,903 commits spread across multiple pull requests. Over the three weeks, the agents developed a growing playbook of refactoring recipes: 22 core patterns plus 82 supporting notes. Some were textbook moves—Extract Function, Guard Clauses—but others emerged as tailored solutions specific to this codebase. Shared Index Range collapsed repeated loops that varied only in their iteration bounds. Action Parameter unified duplicated control flow that differed mainly in which function got called. Uniform Step Table converted ad-hoc function calls into table-driven dispatch.

Model selection mattered significantly. Claude Opus outperformed smaller alternatives and even Anthropic's Codex variant, better capturing emergent patterns and documenting them. Files refactored by weaker models often hit a performance ceiling and could not progress further.

Practitioners Split on What It Proves#

Reaction on LinkedIn split sharply along methodological lines. Supporters, like Paolo Perrone, emphasized the rigor of the verification approach: replaying frame-by-frame traces against a real game set "a much higher bar" than merely passing a green test suite. Mats Iremark reported running a similar experiment and found the combination of CodeScene's tools with modern agents "almost like cheating."