Study reveals LLMs struggle to detect flaws when code changes but documentation stays the same

LLM-powered coding assistants regularly work with multiple sources of information—implementation code, written specifications, method signatures, and test suites—but researchers have limited visibility into how these models weigh conflicting or incomplete evidence. A new study from the Software Engineering community reveals a critical asymmetry: large language models are far better at catching problems in documentation than at recognizing bugs lurking in source code.

The TRACE Evaluation Framework#

Computer scientists have developed TRACE, a rigorous testing methodology designed to measure how LLMs prioritize and trust different software artifacts when those sources disagree. The approach constructs paired versions of real Java method bundles—clean versions alongside deliberately flawed variants where faults are introduced into documentation, code, or both. Researchers then ask models to assess the quality of each artifact, pinpoint inconsistencies, and rank reliability.

The experiment drew responses from seven different LLMs across 456 method bundles, yielding over 22,000 valid assessments. This scale allowed researchers to identify consistent patterns in model behavior across different systems.

Key Findings: A Source-Origin Bias#

The results expose a striking detection gap that researchers term source-origin asymmetry. When documentation contains errors while code remains correct, models catch problems at rates between 67–94%. When documentation and implementation directly contradict each other, detection rates range from 50–91%.

However, the picture reverses when the implementation contains a bug but the documentation is unchanged: performance drops by 21–43 percentage points. Models struggle to flag faulty code when the written specification appears correct and consistent.

Additional Weaknesses#

Beyond the asymmetry problem, the study uncovered two other concerning patterns:

• Weak confidence signals: For six of the seven models tested, the model's reported confidence level provides little separation between correct and incorrect judgments, meaning users cannot reliably use a model's stated certainty as a guide.
• Implementation prioritization bias: Models have difficulty deprioritizing implementations flagged as unreliable, suggesting they treat runnable code as inherently more trustworthy than natural-language specifications.