Intel squeezes ternary AI models smaller than theory predicts by rethinking weight storage

Intel's research team has devised a storage technique that compresses ternary AI models below what was thought to be a fundamental limit—all without altering the model's parameters or retraining. The innovation hinges on a simple insight: real-world ternary models contain far more zeros than the textbook math assumes, and encoding those gaps differently can shrink files and speed up inference.

The 1.58-bit barrier explained#

Ternary language models restrict weights to just three values: −1, 0, and +1. The theoretical minimum to represent three equally probable options is 1.58 bits, derived from information theory. In practice, the standard five-trit-per-eight-bit-byte scheme yields 1.6 bits per weight, or 1.625 when accounting for block alignment in models that group 128 weights together.

This figure assumes weight values occur with equal frequency. That assumption, however, doesn't match what Intel researchers found when they analyzed 29 ternary model checkpoints across seven different model families: zeros ranged from 29.7% to 51.5% of all weights. When a model's zero-weight frequency climbed above 37.5%, a different encoding strategy promised improvement.

BITCOS: separating zeros from signs#

The new BITCOS format (short for "BITmap and COmpacted Signs") splits weights into two independent streams. The first track records a presence bit for every weight—zeroing out whether it is zero or nonzero—while the second stores only the sign bits of nonzero weights. This saves space because zeros need only the presence marker; they carry no sign to encode.

The math is straightforward: if z represents the fraction of zero weights, BITCOS consumes 2 − z bits per weight. At 40% sparsity, that yields 1.6 bits; at 51.5% sparsity, it drops to 1.485 bits. Since unpacking returns the original −1, 0, and +1 values losslessly, model accuracy remains unchanged.

The sparsest checkpoint tested—a ternary Qwen3-1.7B quantized with CAT-Q post-training compression—contained 51.48% zeros. BITCOS compressed it to 1.485 bits per weight, beating both the theoretical minimum and conventional packing.

Decoding performance gains#

Intel developed specialized unpacking kernels for multiple architectures. On CPUs with AVX-512 instruction sets, the presence bitmap acts as a mask for a scatter operation that reconstructs nonzero weights. GPUs lacking that instruction (such as Xe2) use a 2-kilobyte lookup table instead.