Google Brings Edge AI to Raspberry Pi with LiteRT and Gemma Models

Google's AI Edge team has released new capabilities that make it practical to deploy sophisticated AI systems on compact, low-power devices like Raspberry Pi. The tooling eliminates the need for cloud connectivity and enables applications that can see, hear, and respond to their surroundings in real time with complete data privacy.

What's New: LiteRT and Optimized Models#

The approach centers on LiteRT, Google's inference runtime designed specifically for efficient on-device execution. Combined with Gemma, a family of open, lightweight language models, developers now have production-ready tools for autonomous robots, smart cameras, and other edge systems.

The Gemma lineup offers several variants tuned to different hardware constraints:

• Gemma 3 270M — Ultra-compact base model for fine-tuning, suited to sentiment analysis and entity extraction
• EmbeddingGemma 300M — Text embedding model for retrieval-augmented generation and semantic search
• Gemma 3 1B — Multilingual option balancing size with generation quality
• Gemma 4 E2B — Memory-optimized variant designed specifically for mobile and edge environments
• Gemma 4 E4B — High-performance model delivering reasoning capabilities without overwhelming Raspberry Pi's resources

LiteRT-LM, a specialized layer atop LiteRT, handles Gemma deployment out of the box, using CPU acceleration via XNNPACK to maximize efficiency.

Performance: Real-Time Responsiveness on Limited Hardware#

On a Raspberry Pi 5, Gemma 4 E2B achieves 99 tokens per second during the initial prefill phase and 9 tokens per second during decoding, while consuming only 1.4 GB of peak memory. The model's efficient tokenizer averages 4.2 characters per token, translating to roughly 27 characters per second—approximately 300 words per minute—making it suitable for real-time speech and translation tasks.

Architecture: Dividing Work Between CPU and GPU#