Google Axion and GKE Sandbox bring CPU-optimized efficiency to agentic AI

Enterprise teams rolling out autonomous AI agents face a critical infrastructure challenge: how to route diverse workloads across compute resources efficiently. Rather than running everything on expensive accelerators, a smarter architecture leverages high-performance GPUs for training and inference while delegating agent orchestration, API calls, and state management to CPUs—a more economical and secure approach.

Why CPUs excel at agentic workflows#

Processors like Google Axion—the company's first custom Arm-based server CPU introduced in April 2024—are well-suited for the coordination layer that autonomous agents require. These workloads involve managing agent state, semantic routing, selecting the right tools, and spinning up isolated sandboxes to execute untrusted generated code safely.

Bhumik Patel, Director of Software Ecosystem Development at Arm, frames the opportunity clearly: "Agentic tasks such as orchestrating, talking to APIs, and memory management are all ones CPUs are good at, so it's a distributed and concurrent AI workload." By matching workload type to the right compute resource, organizations achieve meaningful cost savings and operational efficiency.

Performance and cost gains#

Google's hybrid execution model shows tangible benefits. The Google Kubernetes Engine (GKE) Agent Sandbox running on Axion-powered N4A instances delivers up to 30% better price-performance than competing hyperscalers for the same workloads, according to Mo Farhat, Axion Group Product Manager at Google. This efficiency stems from reserving accelerators strictly for heavy reasoning and generative tasks while relegating high-concurrency orchestration and context management to Axion CPUs.

Secure execution for untrusted code#

As autonomous agents begin generating and running code dynamically, security becomes non-negotiable. Running AI-generated code directly in a standard cluster risks exposing other applications or the underlying infrastructure to compromise.

The GKE Agent Sandbox addresses this by creating an isolated kernel-level environment with sub-second latency. Built on gVisor—Google's open-source application kernel—the sandbox intercepts and validates system calls before they reach the host kernel. It also supports Kata Containers and default-deny Kubernetes network policies, allowing teams to customize their isolation strategy.