GKE Pod Snapshots Speed Model Startup, Shift Focus to Snapshot Management

Google has unveiled benchmark results for GKE Pod snapshots, showing dramatic reductions in startup latency—large language models loading in 37 seconds for 70B parameters and 15 seconds for 8B parameters. The feature freezes a running container's entire state, including memory and open connections, then restores it on demand rather than requiring full reinitialisation. The capability reached general availability in May on clusters running version 1.35.3-gke.1234000 or later.

How Snapshots Work#

Unlike caching, this is true checkpoint-and-restore. The snapshot captures everything the application had in motion: file handles, running threads, CPU registers, and memory contents, along with the container filesystem and temporary mounts. When a new replica starts from the snapshot, it bypasses the expensive initialization phase—typically model loading—that dominates startup time on large workloads.

The technology relies on gVisor, Google's container sandboxing runtime, which means Pods must run in GKE Sandbox. Autopilot clusters have this enabled by default; standard clusters require a dedicated node pool with gVisor configured. A node-level agent manages snapshot creation and storage in Google Cloud Storage, while a control-plane controller handles cleanup of outdated snapshots. Configuration uses two custom resources: PodSnapshotStorageConfig (to designate the storage bucket) and PodSnapshotPolicy (to target Pods by label, choose trigger types, and set retention rules).

The Compatibility Challenge#

Google's documentation addresses snapshot validity through a "distilled Pod spec"—a hash derived from the Pod's essential runtime fields embedded in each snapshot. Restored Pods must produce an identical hash, and the target node must use the same machine series, CPU architecture, and versions of gVisor and GPU drivers. When no compatible snapshot exists, the Pod simply starts normally.

However, experts have flagged snapshot invalidation as the harder operational problem. Upgrading a node pool can bump the gVisor or GPU driver version, rendering existing snapshots incompatible without warning. The Pod then starts fresh, losing all performance gains silently.

What Happens After Restore#

The application bears responsibility for rehydration. Encryption keys and certificates created before the snapshot must be re-created afterward since the restored process resumes its frozen state. Environment variables, stored in application memory, cannot reliably be found and replaced by gVisor; workloads relying on new values must read them from . External network connections terminate on restore, persistent volumes are not checkpointed, and custom firewall rules do not carry over. Restore is also not instantaneous—gVisor's kernel returns within seconds and the application begins running, but memory loads continue in the background.