AI
Google Releases Gemma 4 12B, a Compact Multimodal Model for Local AI Development
Google's new Gemma 4 12B model runs natively on consumer devices without separate encoders, unifying vision, audio, and text processing in a single inference pass.
Google Developers Blog (AI)
Google is rolling out Gemma 4 12B, a medium-sized multimodal model designed to run locally on standard hardware. The model departs from conventional multimodal architectures by eliminating separate vision and audio encoders—a shift that cuts latency and memory overhead.
The Encoder-Free Approach#
Rather than routing inputs through frozen, multi-stage encoders before reaching the language model backbone, Gemma 4 12B feeds vision and audio directly into its transformer. The vision embedder (35M parameters) projects raw 48×48 pixel patches in a single matrix multiplication, while audio is handled by linearly projecting raw 16 kHz signals into the LLM's input space. This unified design means downstream fine-tuning—via LoRA or full training—updates the entire multimodal token loop in one pass.
Developer-Friendly Specifications#
The model runs on GPU laptops with 16GB VRAM or unified memory, and Google is releasing a dedicated multi-token prediction variant to maximize local inference speed. New macOS desktop applications enable developers to experience fully local audio and visual interaction directly on consumer devices, marking the first time Google has shipped downloadable desktop apps for this use case.
Capabilities & Integration#
Gemma 4 12B handles automatic speech recognition, agentic reasoning, video understanding, and coding tasks. It works seamlessly with existing agent frameworks like OpenCode and can be served locally via llama.cpp or the new LiteRT-LM CLI, which provides OpenAI-compatible API endpoints with stateless prefix caching to bypass prefill latency.
Google is also introducing the Gemma Skills Repository, a library designed to enable agents to build with Gemma models. Weights are available on Hugging Face and Kaggle, with support for popular inference stacks including Hugging Face Transformers, MLX, SGLang, and vLLM. For production deployment, developers can use Google Cloud endpoints, Gemini Enterprise Agent Platform Model Garden, Cloud Run, and GKE.
