Agentic AI
AI coding agents contribute meaningful security work but face stricter code review
A large-scale GitHub study reveals that autonomous agents submit security-focused pull requests at roughly 4% frequency, focusing more on hardening than fixes, yet encounter longer reviews and lower merge rates.
Researchers conducting an empirical analysis of autonomous coding agents on GitHub have found that while these AI teammates are making real security contributions to production code, their work undergoes significantly more scrutiny than typical commits.
What the research examined#
The study analyzed over 33,000 pull requests from the AIDev dataset, drawn from popular GitHub repositories. Investigators used keyword filtering and manual validation to identify 1,293 security-related pull requests submitted by autonomous agents. The analysis then tracked acceptance rates, review timelines, and rejection patterns across different agents and programming ecosystems.
Key findings on agent behavior#
Security-focused contributions account for roughly 4% of all agent-authored pull requests. Notably, most of this work does not involve fixing known vulnerabilities. Instead, agents concentrate on supportive security hardening—tasks like strengthening test coverage, improving documentation, tightening configuration settings, and enhancing error handling throughout codebases.
Review outcomes differ from standard PRs#
Pull requests addressing security consistently encounter longer review cycles and experience lower merge-to-rejection ratios compared to non-security work from the same agents. This pattern suggests that human reviewers apply heightened scrutiny whenever security is involved. However, the research found that PR complexity and verbosity proved stronger predictors of rejection than explicit security focus—indicating that how agents present changes matters as much as what they change.
Implications#
The findings suggest that autonomous coding agents have established themselves as legitimate contributors to software security workflows, though not necessarily in the ways conventional wisdom might expect. Their evolution beyond simple vulnerability patching toward comprehensive hardening reflects the nuanced security work that modern development actually demands.
