GraftyVul Creates Testable Security Datasets by Transplanting Real Vulnerabilities into Code

Security researchers need robust datasets of vulnerable code to train detection tools and test remediation systems. Yet most existing vulnerability datasets force researchers to choose between breadth, reliability, or real-world relevance—rarely do they offer all three.

A new framework called GraftyVul attempts to bridge that gap by taking documented vulnerabilities from live systems and deliberately introducing them into open-source projects in controlled ways.

How It Works#

The approach starts with vulnerabilities that have already appeared in real code. Rather than simulating or synthetically generating flawed code, GraftyVul grafts these known weaknesses into existing open-source repositories. Because the host projects come with their own build systems and test suites, researchers can verify that each injected vulnerability actually changes program behavior—they can run exploit scripts that confirm the vulnerability is genuinely exploitable.

This combination of realism (the vulnerability is from a real context) and reproducibility (you can build, test, and verify it yourself) is what sets the approach apart.

The Dataset#

Using this method, the team created 212 distinct vulnerable programs across five languages—Python, TypeScript, Java, Go, and C#—spanning 23 different weakness categories (CWE types). Each one has been verified to be genuinely exploitable.

To assess how well these synthetic samples resembled actual vulnerabilities, the researchers developed a language-agnostic embedding technique. Rather than comparing code line-by-line, this method captures the semantic essence of a vulnerability: where it happens (sink), how it works (mechanism), and what system features it exploits. By this measure, GraftyVul samples retained a strong connection to their source vulnerabilities.

Practical Impact#

When compared head-to-head with 13 widely used vulnerability datasets, GraftyVul achieved competitive coverage of languages and weakness types while being the only dataset that offers both reproducible exploits and broad scope. An industrial case study showed the dataset's value in evaluating production systems designed to automatically fix security flaws.