AI
Google brings high-performance AI inference to the web with LiteRT.js
A new JavaScript runtime lets developers run machine learning models directly in browsers with hardware acceleration, matching native app performance.
Google Developers Blog (AI)
Google has released LiteRT.js, a JavaScript framework that enables machine learning models to run natively in web browsers with near-native performance. The tool is built on Google's established LiteRT platform, which currently powers on-device AI across Android and iOS.
The release addresses a long-standing limitation in web development: previous solutions for browser-based AI inference relied on JavaScript-based computation kernels that were significantly slower than native implementations. LiteRT.js bypasses this bottleneck by exposing LiteRT's optimized runtime directly to web developers through WebAssembly, unlocking access to the same hardware accelerators that mobile and desktop apps use.
What developers can do with it#
Web developers working in JavaScript or TypeScript can now handle AI tasks entirely on the client side—including text generation, object detection, audio transcription, and image processing. Because computation stays local, applications built with LiteRT.js gain three major advantages: enhanced user privacy (no data leaves the browser), zero server infrastructure costs, and minimal latency for interactive experiences.
Existing developers familiar with .tflite model formats can treat LiteRT.js as a more capable successor to TensorFlow.js. The framework handles model conversion from popular Python machine learning frameworks and automatically applies the latest performance improvements, quantization schemes, and hardware optimizations.
Hardware acceleration across devices#
LiteRT.js supports multiple execution paths depending on device capabilities:
- CPU acceleration: Uses XNNPACK, Google's optimized library for on-device processing, with multi-threading and advanced SIMD support.
- GPU acceleration: Leverages WebGPU to enable state-of-the-art GPU computation in browsers.
- NPU support: Targets dedicated neural processing units via the emerging WebNN API (currently experimental in Chrome and Edge), enabling ultra-low-power inference on compatible hardware.
According to Google's benchmarks conducted on a 2024 Apple MacBook Pro with M4 processors, LiteRT.js outperformed other web AI runtimes by up to 3x on CPU inference. For GPU-accelerated tasks, switching from CPU to GPU or NPU backends delivered speedups between 5–60x, depending on workload complexity.
