AI Infrastructure
Running AI agents on Kubernetes: shift costs from idle waiting to active work
Google's new Agent Substrate approach lets coding agents maintain persistent sessions while freeing compute resources when idle, cutting infrastructure costs by up to 40%.
Google Developers Blog (AI)
When developer-facing AI agents move from laptops to shared Kubernetes clusters, the runtime economics change fundamentally. A session-based workload—where an agent stays open all day but works only when prompted—demands a different cost model than traditional web applications.
Why sessions aren't requests#
Web applications handle discrete requests that last milliseconds: a server processes each one and discards its state. Kubernetes responds by scaling pods based on incoming traffic volume. Coding agents work differently. A developer's session persists throughout the workday, maintaining memory and file state between prompts while remaining idle most of the time.
Recent research on GitHub Copilot agent traces reveals that agents spend roughly 80% of their session time waiting for the next user input. With typical usage of 2 prompts per hour over an 8-hour workday, an agent might work actively for just 22% of the day—yet consume a full machine's resources for the other 78%.
Two traditional approaches—and their cost limits#
Teams running agents on Kubernetes typically choose between two strategies, both familiar from web-scale deployments:
- One pod per session keeps state intact but dedicates resources all day. A
c3-standard-4 nodefits 3 such pods after system reservations, meaning 100 coding agents require roughly 34 nodes and cost about USD 1,288 monthly—even when idle. - Scale to zero frees the machine between prompts but loses the running process and in-memory state. Resuming requires starting fresh on the next request.
Parking: keeping the session, freeing the machine#
Google's Agent Substrate and AX frameworks introduce a third model: maintain the agent session while returning compute to the cluster when idle. This "parking" approach snapshots an idle actor to object storage and restores it on a warm worker when the next prompt arrives, allowing multiple agents to share the same underlying compute.
Agent Substrate handles the runtime layer—where and how agents run. It isolates each agent in a sandbox (called an actor) that holds running processes, files, and credentials. Workers are reusable sandbox pools drawn from Kubernetes pods; one worker can host multiple parked actors. defines what the agent is and what it knows, managing tasks (what runs) and workspaces (repositories and files).

