AI Vulnerability Detectors Flip Predictions on Minor Code Changes, Study Finds

A new study reveals a significant fragility in machine learning models trained to detect code vulnerabilities: small, meaning-preserving edits can push security predictions across the line from flagging a bug to clearing it—even when the underlying flaw remains unchanged.

The Problem: Embedding Drift#

Researchers tested how frozen code embeddings—numerical representations generated by pre-trained models like CodeBERT—respond to common transformations that don't alter a program's actual behavior. The investigation used 7,500 paired C/C++ functions (both vulnerable and patched versions) from the Big-Vul dataset, applying four types of benign edits: comment removal, unreachable code insertion, variable renaming, and loop rewriting.

The results were sobering. Under all transformations applied together, the baseline rate at which correctly flagged vulnerabilities were misclassified as safe ranged from 35.55% to 42.15% across six different downstream classifiers. Such instability is deeply problematic for security tools, since developers might introduce trivial code changes without realizing they've inadvertently hidden a vulnerability from automated detection.

A Partial Defense#

The researchers proposed a training-time mitigation that pushes the embeddings of clean code and their mutated variants closer together in representation space, while preserving information about whether code is vulnerable. This approach reduced the average misclassification rate under combined mutations from 22.60% to 11.32%, and improved accuracy on mutated samples from 55.33% to 57.10%.

However, the defense introduced its own trade-off: while it reduced false negatives (vulnerabilities missed), it increased false positives (benign code flagged as vulnerable)—by as much as 11.95 percentage points for one classifier. The technique also did not fully eliminate prediction instability, suggesting that deeper architectural changes may be needed.

Why It Matters#

The findings underscore a critical gap between how humans understand code and how machine learning models process it. Semantics-preserving transformations should be "invisible" to security tools, yet they consistently cause misclassifications. As automated vulnerability detection becomes more prevalent, ensuring robustness to such transformations is essential to prevent false confidence in buggy code.