Google Launches AI Agent Safety Layer to Catch Risky Behavior Before It Causes Damage

As AI agents take on more autonomous business tasks—processing refunds, updating records, calling internal tools—the risks they pose shift from code vulnerabilities to behavioral ones. A session might appear successful on the surface while the agent quietly accesses tools it shouldn't or escalates its own permissions. Google has introduced a new solution to catch these hidden problems.

The Problem With Silent Failures#

Autonomous agents can complete tasks flawlessly according to technical metrics while still violating security policies or business rules. The challenge is that these behavioral anomalies often go undetected because no system fails—the agent simply answers and closes the ticket. Only later does a security audit reveal the agent reached for unauthorized tools or acted on requests that shouldn't have been granted. Traditional monitoring focused on error rates and performance misses these subtler threats entirely.

Introducing Agent Anomaly Detection#

Google is now offering Agent Anomaly Detection in private preview on its Gemini Enterprise Agent Platform. The system acts as a reasoning-based oversight layer that examines agent behavior through reasoning traces, tool calls, and execution flows across entire sessions.

The detector ingests logs and OpenTelemetry traces agents already emit, then evaluates whether the agent is operating within its intended boundaries. When it finds suspicious patterns, it flags policy violations, unintended tool use, and potential escalations of privilege.

How It Works: Layered Analysis#

Detection runs in layers to balance speed, cost, and accuracy:

• First pass: A lightweight statistical scanner reviews all traffic to identify outlier sessions and surface volume-based anomalies.
• Second pass: An LLM-based reasoning engine deeply examines flagged sessions, reasoning through the full context of user requests and agent actions.
• Third pass (when needed): Detailed reconstruction of individual tool calls and their parameters for precise accountability.

Consider an inventory agent that lists items in bulk. A user asks the agent to "list 100 items at a time," and the agent pages through large batches by jumping across database offsets to pull the entire catalog. No error occurs—the agent only uses tools it has permission to use. But Agent Anomaly Detection recognizes the pattern as systematic scraping rather than normal browsing and flags it as with critical severity.