Security & AI
AI Framework Automatically Writes Security Vulnerability Detectors at 53% Accuracy
Researchers built an agentic system that uses language models with code execution feedback to generate security analysis queries automatically from CVE records.
Finding security bugs in code requires writing specialized queries that recognize vulnerable patterns. This typically demands deep expertise in both security and static analysis tools—a combination few developers possess. Researchers have now created a system that automates this work by having AI synthesize the queries directly from publicly available vulnerability data.
The Problem with Manual Query Writing#
Static analysis tools like CodeQL are highly effective at catching security flaws, but only when security experts manually craft queries that match specific vulnerable code patterns. Writing these queries demands specialized knowledge of both the analysis engine's query language and the subtle signs of exploitable code. This bottleneck limits how quickly organizations can detect emerging vulnerability classes.
How QLCoder Works#
The new system, called QLCoder, takes a CVE vulnerability description as input and automatically generates CodeQL queries that detect the flaw. The approach embeds a large language model inside a loop that runs the generated queries against actual code and learns from the results. To keep the system on track, QLCoder constrains the LLM's reasoning using a custom interface that provides two key resources:
- Syntax guidance via a Language Server Protocol connection, ensuring queries follow correct grammar
- Semantic grounding through a retrieval-augmented generation database that pulls relevant past queries and documentation
This feedback-driven process allows the system to refine its attempts and avoid common mistakes.
Results and Comparison#
In testing across 176 real CVEs in Java projects, QLCoder achieved a 53.4% success rate—generating queries that correctly identified the vulnerability in buggy code but not in patched versions. By contrast, using Claude Code alone (without the structured feedback and constraints) produced correct queries only 10% of the time. The research team built QLCoder on top of the Claude Code agent framework, and the code is publicly available for others to build on.
