New Method Cuts Visual Document Storage Needs by Up to 155x Using AI Embeddings

A team of researchers has developed CausalEmbed, a technique that dramatically shrinks the storage footprint required for visual document retrieval systems without sacrificing search quality.

The Storage Problem#

Multimodal large language models (MLLMs) can extract detailed information from documents and images to improve search functionality, but they typically require representing each page using thousands of visual tokens. This creates a significant practical barrier—the sheer volume of data becomes unwieldy for real-world deployment. CausalEmbed tackles this bottleneck by generating multi-vector embeddings through an auto-regressive process, substantially reducing how many tokens are needed to represent a document.

How It Works#

The approach combines an iterative margin loss technique during contrastive training, encouraging the embedding model to develop tightly packed and well-organized representations. By adopting this generative strategy, CausalEmbed achieves a 30-155x reduction in the number of visual tokens required per page while preserving competitive performance across different model architectures and benchmark datasets.

Broader Implications#

Beyond efficiency gains, the method introduces a flexible test-time scaling approach for multi-vector document retrieval, pointing toward a generative paradigm that could reshape how multimodal document systems are designed. The research demonstrates theoretical advantages in training speed and scalability, suggesting this direction may unlock new possibilities for deploying advanced search systems in resource-constrained environments.