AI Security
New Framework Exposes Security Gaps in AI Agents Operating with System Privileges
Researchers developed ASEval, an automated testing system that reveals how autonomous agents can be manipulated through multi-turn conversations to bypass safety measures.
Autonomous agents built on large language models are increasingly granted deep system-level access to complete complex tasks, yet traditional safety testing methods fail to catch vulnerabilities specific to how these agents actually operate. A new research framework aims to change that.
The Testing Gap#
Current LLM safety evaluations focus on single prompts and immediate responses, missing the sequential and stateful nature of real agent operations. When autonomous systems can chain multiple actions, maintain context across turns, and interact with external tools—as modern agents do—existing testing approaches leave significant blind spots. Security flaws that manifest across multiple steps or emerge from how agents handle persistent state remain largely undetected.
ASEval's Approach#
Researchers introduced ASEval, an automated framework that generates realistic multi-turn test scenarios and transforms them into security tests. The system works by creating benign conversations, applying carefully crafted perturbations to inject risks, running these cases against target agents, and evaluating the complete sequence of actions taken rather than just the final response.
The team built A3S-Bench, a test suite comprising 2,254 executable cases covering diverse usage patterns and security risk categories. Testing across 11 agent variants revealed dramatic improvements in detection: their perturbation operators raised the average risk-trigger rate from 22.9% under conventional injection techniques to 47.4%.
Key Findings#
The evaluation showed that an action-grounded oracle—a system that judges whether agents actually took harmful actions—aligns with human judgment at 93.2% binary F1 score. More importantly, the analysis uncovered that many security failures emerge from characteristics that response-only evaluation misses: multi-turn delivery dynamics, persistent state handling, and tool-mediated actions that intermediate steps might obscure.
These findings suggest that robust agent security requires testing frameworks that track behavior across entire execution traces, not just isolated inputs and outputs.
