Google's Custom Gemini Chip Bets on Model-Specific Hardware for Cheaper AI Inference

Google is reportedly developing a custom silicon processor specifically optimized for its Gemini AI model—a move that underscores a wider industry pivot away from generic computing hardware toward inference chips tailored to individual models. The project, codenamed "Frozen v2" and first reported by The Information, represents an unconventional middle ground: hardwiring parts of Gemini's architecture into the chip while keeping the model's weights updatable.

The Economics of Specialized Hardware#

The push to reduce AI inference costs is fundamentally reshaping chip design. Unlike general-purpose accelerators such as Nvidia GPUs or Google's own Tensor Processing Units—which must accommodate a wide variety of models and workloads—specialized silicon can be optimized for a single architecture. Early projections suggest Google's chip could deliver six to ten times more tokens per watt than its current AI processors.

This mirrors patterns seen elsewhere: Bitcoin mining hardware followed a similar trajectory, evolving from CPUs to GPUs to application-specific integrated circuits (ASICs). As AI systems mature from research to production, the flexibility needed during training becomes less critical than maximizing efficiency and throughput during inference.

Balancing Efficiency and Evolution#

The core challenge lies in avoiding hardware obsolescence. An earlier concept reportedly would have embedded Gemini's weights directly into the silicon—the ultimate in optimization—but that approach proved unworkable. Once the model parameters were baked into the chip, updating or improving Gemini would require replacing the entire processor.

Frozen v2 sidesteps this trap by freezing only the underlying computational architecture while allowing weights to be refreshed over time. Google engineers have spent years calibrating this balance, according to reporting from The Information. The result: hardware that gains much of the efficiency of model-specific silicon without becoming obsolete with each Gemini iteration.

A Broader Industry Shift#

Google is not pioneering specialized inference hardware alone. Companies like Canadian startup Taalas have demonstrated chips with entire 8-billion-parameter models embedded in silicon, reportedly achieving throughput of roughly 17,000 tokens per second by eliminating frequent data transfers between memory and compute. Nvidia itself, which dominates AI processors, has invested heavily in inference optimization through licensing deals, including a reported $20 billion arrangement with Groq last year.