Agentic AI
Google Releases AQuA, Tool to Catch Silent Failures in Production AI Agents
A new open-source sidecar agent monitors deployed AI systems for hard-to-catch quality regressions missed by traditional monitoring.
InfoQ
Google has made available Ambient Quality Agent (AQuA), an open-source reference tool designed to identify and diagnose failures in production AI agent deployments that conventional monitoring systems typically overlook—such as agents disregarding user settings or omitting critical tool calls while still reporting success.
How AQuA Works#
AQuA operates as a sidecar component alongside agents running on Google Cloud, sitting outside the request handling path and consuming telemetry from Cloud Trace, Cloud Logging, or BigQuery. The tool processes sessions through a five-stage pipeline: sampling (drawing up to 1,000 recent sessions), reviewing their outputs, grouping related issues together, validating clusters against supporting evidence, and maintaining an ongoing record of problems. Teams can define domain-specific quality expectations using a goal.md configuration file and plug in custom deterministic metrics. The system can execute on a regular schedule, immediately after a deployment, or when manually triggered.
Root Cause Investigation#
When failures occur, AQuA compares broken sessions against an immutable snapshot of source code captured at the time of deployment. A diagnostic agent scans relevant code and configuration, cites specific file locations and line numbers, and suggests modifications when the problem originates from the repository itself. The system can also pinpoint external factors—third-party service failures, tool integration issues, or problematic retrieved data—as responsible for problems. Importantly, AQuA does not automatically apply fixes or create pull requests; it surfaces findings for human review.
Real-World Results#
In testing against a multi-agent travel planning application, AQuA examined 32 sessions and surfaced 42 findings that clustered into nine candidate problem areas. The verification step discarded three clusters lacking sufficient supporting evidence, confirming six genuine issues. Two major categories emerged: agents bypassing seat availability validation and losing traveler dietary preferences during handoffs to other agents.
After engineers updated prompts and re-tested those same sessions, seat-selection bypasses dropped from 15 to 2, preference omissions fell from 7 to 0, and overall session success rates climbed from 5 to 13 passing all checks. These metrics reflect controlled replay rather than live traffic behavior.

