Google Opens HEIR, a Compiler Toolkit to Run AI Models on Encrypted Data

Google has unveiled HEIR (Homomorphic Encryption Intermediate Representation), an open-source compiler platform designed to simplify the deployment of encrypted computation across machine learning systems. The toolchain enables developers to convert existing AI models—originally built for unencrypted data—to instead work directly on encrypted inputs, opening new possibilities for privacy-preserving inference.

What the technology does#

Homomorphic encryption fundamentally shifts how sensitive data can be handled. Rather than decrypting information to process it, servers can perform calculations directly on ciphertexts and return encrypted results, keeping the underlying data hidden throughout. This approach addresses several real-world challenges: spam detection and malware scanning can run on encrypted user data, deployed models remain unexposed on user devices, and cloud services can detect fraud or intrusions without exposing proprietary logic or sensitive details.

HEIR abstracts away much of the complexity by acting as a bridge between standard machine learning code and encrypted execution. Developers write their logic in Python, flag which variables should be encrypted, and let HEIR handle the compilation pipeline. Google has demonstrated the approach across use cases including private content recommendations, credit card fraud detection, network intrusion identification, and audio hotword recognition—all without revealing sensitive data to the service.

How it works in practice#

The current workflow requires several manual steps. Developers typically export PyTorch models to MLIR (a compiler intermediate representation) using torch_mlir, then feed that into HEIR for further optimization and FHE compilation. Google notes that full automation remains a goal rather than today's reality.

The performance question#

A significant debate surrounds whether homomorphic encryption can be commercially practical. Community observers point to substantial overhead—typical estimates range from 1000x slowdown on certain operations. Specific examples cited include 80 milliseconds for 64-bit equality checks, 100 milliseconds for addition or subtraction, and up to 8 seconds for division.

However, some researchers suggest the constraints may be less severe for large language models. Since LLM inference relies heavily on matrix multiplication and addition—operations where FHE performs relatively well—and avoids expensive branching logic, the penalties might be more manageable in that context. Others argue the trade-off may be worth it for specific applications: a one-millisecond image classification task that balloons to one second could still be acceptable for privacy-critical scenarios.