Agentic AI
AI Needs Human Experts to Uncover Truly Novel Hacking Techniques
Security researcher James Kettle's experiments reveal that while AI can accelerate vulnerability discovery, the most powerful cybersecurity breakthroughs still require human judgment to synthesize AI findings into actionable attack strategies.
WIRED — AI
Artificial intelligence has transformed the speed of vulnerability discovery and exploit development, yet a new investigation suggests that truly innovative hacking techniques remain beyond the reach of autonomous AI systems—at least for now. Security researcher James Kettle presented his findings at Black Hat Las Vegas this week, painting a more nuanced picture of AI's role in cybersecurity.
The Research Question#
Kettle set out to test whether agentic AI could autonomously conceive and execute entirely novel attack strategies, moving beyond incremental improvements to vulnerability patching. His months-long experiments, conducted using recent models from Anthropic and OpenAI, yielded a conclusion with important implications: AI struggles when asked to operate in full autonomy but becomes extraordinarily effective when paired with human expertise at critical junctures.
How AI Accelerated Discovery#
Rather than pursuing theoretical research where AI might fabricate findings, Kettle scoped his work to his own area of specialization—web security—ensuring he could verify results. As he fed models his research methodology and increasingly refined parameters, the pace of discovery accelerated dramatically. Where Kettle might uncover significant findings over weeks, the AI-assisted workflow produced multiple leads every few days.
"It was making me anxious," Kettle recalls, describing the volume of potential research paths the system generated. This volume pushed him to automate further analysis, creating a feedback loop that proved more productive than his individual efforts.
The Breakthrough: Shared-Parser Confusion#
The most significant discovery emerged from human-AI collaboration rather than pure automation. Kettle's systems analyzed proven vulnerability findings and formulated a hypothesis about Shared-Parser Confusion—an entirely new vulnerability category stemming from how web servers use shared code to process both incoming requests and outgoing responses.
The implications are substantial. Untrusted request traffic potentially opens a vast attack surface that could enable multiple exploit types. Yet Kettle emphasizes that the AI couldn't validate this insight independently. "It wasn't able to prove this itself," he explains, "but it analyzed some real, proven findings and came up with the hypothesis, and I evaluated it and confirmed it."
