LLM code passes tests but fails in production due to structural inconsistencies

Large language models can produce code that compiles cleanly, passes unit tests, and looks syntactically correct—yet still breaks catastrophically in production. A new research paper from arXiv identifies the fundamental problem: AI-generated code frequently suffers from what researchers call the patchwork problem, where individual code segments are locally coherent but globally incoherent.

The core issue#

The failures are structural rather than logical. An API endpoint might reference configuration keys that were never declared anywhere in the project. An import statement could target a package that doesn't exist in any package registry. A newly created route might lack the authentication middleware that guards every other similar endpoint. Each piece, examined in isolation, appears correct—but the code can't function when assembled with the rest of the system.

Standard continuous integration toolchains and type checkers routinely miss these failures. As organizations increasingly adopt LLM-powered coding assistants, this blind spot poses a growing threat to software quality.

Research approach#

The researchers formalize the problem by treating structural coherence as consistency invariants that span multiple graph representations of how code and configuration fit together: import graphs, call graphs, dependency graphs, configuration graphs, schema graphs, resource graphs, control-flow graphs, and routing graphs.

The team proposes an eight-category failure taxonomy that distinguishes defects unique to LLM generation from those merely made more frequent by it. Their verification framework combines existing mature static analysis tools with custom detectors built to catch cross-cutting invariants that traditional toolchains leave unguarded.

Key findings#

Testing across two state-of-the-art language models and four prompting strategies revealed striking patterns:

• The vast majority of structural failures remain undetected by type checking, unit testing, and static application security testing (SAST) tools
• Failure modes differ qualitatively between different models, complicating attempts at model-agnostic fixes
• These problems persist in real-world repositories where LLMs generated code with minimal human review—they're not artifacts of controlled lab experiments

The research underscores that as AI coding assistants become mainstream, organizations need new verification strategies specifically designed to catch structural inconsistencies that traditional software quality gates are not equipped to detect.