Researchers Cut FastText Memory Use by 80% With Trie-Based Compression

FastText has long been valued for its ability to generate word vectors for unseen terms by decomposing them into character n-grams. However, the original implementation's reliance on hash-bucket indexing creates a scaling bottleneck: collisions between unrelated n-grams waste memory, and expanding the hash table quickly becomes the dominant storage cost.

A Collision-Free Alternative#

Researchers have introduced a redesigned FastText variant that ditches hash buckets in favor of double-array trie structures, a data organization method that guarantees each n-gram receives a unique identity without collisions. Rather than hashing everything and hoping for minimal conflicts, the new approach first assigns every observed word and n-gram an explicit index, then selectively compresses only those rows whose learned vectors justify sharing.

The key innovation is combining structurally constrained prefix and suffix merging with mark-compact memory reorganization. Unlike arbitrary hash collisions, rows are shared only when their vector representations exhibit high cosine similarity—a meaningful similarity threshold rather than a storage accident. This preserves interpretable n-gram identities while reducing the total number of active rows in memory.

Handling Real-World Scale#

The pipeline extends to the full training and serving cycle: UTF-8 aware n-gram enumeration, double-array trie lookup, memory-mapped model loading, and vector reconstruction for both individual words and sentences. On a benchmark with over 30 million Chinese words and 287 million extracted n-grams, the compressed model shrinks from 145.2GB to 28.9GB—roughly an 80% reduction. Load time drops from 12.3 minutes to 3.2 minutes, and downstream task quality remains within 0.3 points of an uncompressed baseline.

Broader Implications#

The authors position this advancement as a compact lexical memory layer for retrieval systems in the large language model era, where efficient vocabularies matter for both inference speed and resource consumption. The implementation has been released as an extended FastText prototype, making it available to researchers and practitioners working with large-scale text embeddings.