LLM-based code review tools can be tricked into missing security flaws via metadata manipulation

Automated code review systems powered by large language models are becoming standard in software development pipelines, but a fresh research paper reveals they have a critical weakness: they can be manipulated into ignoring security vulnerabilities through context crafting.

The Framing Effect in AI Code Review#

The researchers investigated a well-known cognitive bias called the framing effect—where the way information is presented influences judgment more than the substance itself. They found this bias is not just a human problem; LLM-based vulnerability detection also falls victim to it. Across six different models tested under five different contextual framings, the team demonstrated that this is a systematic and reproducible phenomenon across the board.

Attacking Real-World Pipelines#

The team conducted experiments on 33 real-world security vulnerabilities (CVEs) across 20 actual software projects, targeting two widely used code review tools: Claude Code and CodeRabbit. They tested two distinct attack methods:

• Template-based attacks: Crafted PR descriptions and metadata designed to bias reviewers away from finding flaws. These proved mostly ineffective and sometimes counterproductive, as obvious manipulation raised red flags.
• LLM-assisted refinement attacks: Using a local copy of the review pipeline, attackers iteratively refined attack prompts to find the most effective wording. This approach succeeded in 32 out of 33 cases (97%).

Why Refinement Attacks Work#

The researchers identified a fundamental asymmetry that attackers can exploit: adversaries can test attacks repeatedly against a cloned version of the system, learning what works before deploying in the wild. Defenders, by contrast, only get one opportunity to catch attacks in production. This imbalance makes the defender's job substantially harder and gives attackers a decisive advantage in the cat-and-mouse game.

Broader Implications#

The findings underscore a troubling reality about over-relying on automated security tools. While LLM-assisted code review offers efficiency gains, the research highlights that these systems can be gamed and should never replace human judgment entirely. The study emphasizes the need for human oversight and verifying that contributors are trustworthy—technological safeguards alone are insufficient.