AI Security
All Major AI Code Generators Produce Vulnerable Automation Scripts, Study Finds
A comparative analysis of ChatGPT, Copilot, and Gemini reveals that every generated automation script contained exploitable security flaws, regardless of which model was used.
Researchers have demonstrated a consistent and troubling security risk across the leading large language model-based code generation tools: every single script generated for automation tasks contained at least one exploitable vulnerability, according to a new cross-model study.
The Study Design#
The analysis examined code produced by ChatGPT, Microsoft Copilot, and Google Gemini in response to identical prompts spanning three automation domains. Each script was then subjected to a standardized vulnerability review conducted by Claude Code. Researchers assigned severity ratings using the CVSS v3.1 framework and cross-referenced findings against industry standards including the OWASP Top 10:2021 and MITRE ATT&CK taxonomies.
Key Findings#
The results paint a sobering picture of security across the board:
- 100% vulnerability rate: Every generated script contained exploitable flaws
- Consistent patterns: Nine of the seventeen vulnerability classes detected appeared in code from all three models
- Wide coverage: Fourteen of seventeen vulnerability classes showed up in at least two different models
- Similar severity levels: Weighted CVSS scores differed by less than 10% across the three platforms
Implications for Enterprise Deployment#
The study challenges the assumption that security risks in AI-generated code stem from any particular tool's design or training. Instead, the evidence suggests the problem is fundamentally tied to the nature of the automation tasks themselves. This finding shifts responsibility away from vendor selection toward deployment practices: the critical question is not which coding assistant to trust, but rather whether scripts generated by any of these tools should enter production workflows without rigorous human security review.
For organizations relying on non-expert developers or rapid automation deployment, the research underscores the need for mandatory code review and security scanning before these systems integrate with enterprise infrastructure.
