Study reveals severe security gaps in AI coding agents—up to 96% exploitable via malicious skill files

Autonomous code-generation agents now commonly run within enterprise environments with real authority over connected systems, but a new study has uncovered a critical vulnerability in how they load and execute custom instructions.

The Attack Surface#

Coding agents rely on a "skills interface"—directories of scripts and natural-language instructions that agents dynamically fetch to adapt their behavior. Researchers found that this design opens a dangerous door: malicious shell commands can be disguised as harmless skill files, allowing attackers to trick agents into executing arbitrary code.

Research Findings#

Researchers from across academia synthesized adversarial skill files using six large language models to transform 471 real-world shell commands into deceptive instructions. The resulting benchmark contains 2,826 distinct skills mapped to 11 MITRE ATT&CK attack tactics—a standardized framework for categorizing adversarial techniques.

The team then tested two widely-used enterprise-grade agents:

• Google Gemini CLI: Fell victim to malicious skills in 95.5–96.1% of test runs
• Alibaba Qwen Code: Exploited in 71.6–74.0% of runs

These failure rates remained consistent regardless of which language model originally generated the deceptive skills, suggesting the vulnerability cuts across different systems and approaches.

Evaluation and Validation#

The researchers developed a rigorous assessment pipeline involving multiple validation layers: evidence anchoring, refusal checks, and a three-judge LLM panel to render verdicts. Their methodology was cross-checked against human reviewers achieving 85% inter-rater agreement (Cohen's kappa = 0.85), lending credibility to the results.

Notably, explicit safety detection—where agents recognize and reject dangerous instructions—occurred in only about 2% of runs, indicating that current safeguards are largely ineffective.

Implications#

The study points to a significant risk for organizations deploying coding agents in production. Enterprises must evaluate and address skill-interface vulnerabilities before widespread adoption. The authors have released their code and dataset for further research.