Agentic AI
Critical Control Failures Found in Popular LLM Agent Frameworks
A new study reveals that widely-used agent control systems fail to enforce their safety promises, with frontline models triggering dangerous execution leaks in production environments.
Production LLM-agent frameworks advertise safety features—human approval gates, run cancellation, and execution timeouts—that are supposed to act as hard stops. But new research shows none of the six major open-source frameworks actually enforce these promises as documented.
The Gap Between Promise and Practice#
Control primitives in agent frameworks carry names and documentation that suggest ironclad protection: when a run pauses for approval, gets cancelled, or times out, no gated side effects should execute. In reality, researchers found the contract breaks down across every major implementation. Model-free differential probes identified a recurring flaw called "sibling leak"—when an approval gate suspends one branch of execution, a related branch's effects still run during the pause, effectively defeating human rejection. This vulnerability appears in five of six tested frameworks, across four different execution models and two language runtimes.
Beyond sibling leaks, the study documents additional failure modes: duplicate executions on replay, orphaned processes after cancellation, and zombie processes that continue running after timeouts expire.
Real-World Risk and Frequency#
The hazard isn't merely theoretical. Frontier-class language models emit the plan shapes that trigger leaks at rates up to 14%. When live models drive unmodified frameworks, actual leaks occur in 215 of 1,200 test runs, with a 100% leak rate whenever the vulnerable pattern appears. While naturalistic benchmarks show models tend to serialize writes—keeping the latent gap hidden in everyday use—targeted injection reveals the failure is deterministic and reproducible. A public corpus of 13 incidents corroborates the replay and cancellation failures in the wild.
A Verified Solution#
Researchers developed SOUNDGATE, an environment-external gate written in Rust that mediates every side effect through a formally verified control point. The system enforces:
- Hold-until-decided semantics for approvals
- Reject-cancels behavior on refusal
- Deduplication on replay
- Fencing on cancellation
The admission core underwent mechanical verification using Verus, TLA+/TLC (tested to 7.5 billion states), TLAPS, and Loom testing on the deployed Rust code. Differential conformance testing across 12 million operations confirmed zero divergence from specification.
