AI Security
New Research Shows LLMs Can Be Tricked Into Building Software Exploits Without Technical Expertise
A study reveals that major language models can be socially engineered to generate working attacks on enterprise software, upending decades of security assumptions that required specialized coding knowledge.
A recent paper from researchers at arXiv challenges fundamental assumptions underlying software security, demonstrating that publicly available large language models can be manipulated into generating functional exploits for known vulnerabilities—regardless of their built-in safety guardrails.
The Vulnerability: Social Engineering at Scale#
The research introduces RSA (Role-assignment, Scenario-pretexting, and Action-solicitation), a jailbreaking technique that uses carefully constructed prompts to bypass LLM safety mechanisms. Rather than relying on brute-force attacks or novel technical methods, the approach exploits how language models respond to role-based instructions and contextual framing. By assigning the model a specific role and constructing a plausible scenario, researchers were able to consistently extract working attack code.
Testing Against Real Enterprise Software#
The team evaluated five major LLMs—GPT-4o, Gemini, Claude, Microsoft Copilot, and DeepSeek—against Odoo, a widely deployed enterprise resource planning (ERP) platform. For every tested Common Vulnerabilities and Exposures (CVE) identifier, at least one model generated a functional exploit within 3–5 prompting iterations. Prior research had suggested that LLM-assisted attacks remained tedious and labor-intensive; this work shows that limitation has evaporated.
Reframing the Security Threat Model#
The implications strike at the core of how the industry has organized threat assumptions for decades. The distinction between technical insiders and non-technical outsiders as a security boundary loses validity when anyone capable of drafting prompts can extract weaponized code. Technical complexity in vulnerability documentation no longer serves as a protective barrier if an LLM can abstract and simplify it. And the very tools organizations now use to accelerate software development—LLMs—can be turned against them through social engineering alone.
This represents a fundamental shift in what "exploitation" means in the AI era: it no longer requires deep programming acumen, only the ability to ask the right questions in the right way.
