AI
Cohere's new Fast embedding model maintains quality while cutting query costs and latency
Two-tier embedding approach lets teams index with Cohere's premium model and query with a faster, cheaper alternative without re-indexing data.
The New Stack
Cohere unveiled Embed 5, a dual-model embedding system that decouples indexing from querying, allowing organizations to optimize for both retrieval quality and performance. The Pro version handles expensive indexing operations while the Fast variant processes queries at 2.4 times the throughput at a lower cost.
A shared embedding space#
The key innovation behind Embed 5 lies in its unified vector space. Unlike traditional systems that require separate indexes for different embedding models, Cohere's Pro and Fast variants produce compatible vectors at identical dimensions. This means teams can re-index once with Pro, then freely switch to Fast for queries—or even mix approaches—without regenerating the entire corpus.
Pricing reflects the performance split: Pro costs $0.12 per million tokens, while Fast drops to $0.08. For typical RAG and agent workflows where documents are indexed infrequently but searched repeatedly, this architecture delivers meaningful cost and latency savings on the query path.
Quality tradeoffs remain modest#
Cohere's internal testing across 40 datasets suggests the speed-quality tradeoff is manageable. A Pro-indexed corpus queried with Fast achieved a 98.4 retrieval score relative to a Pro-to-Pro baseline of 100—a gap of just 1.6 points. Using Fast for both indexing and querying fell to 96.6, and the company reports no individual dataset showed dramatic performance degradation with either configuration.
However, these results average across Cohere's own benchmarks; production systems handling customer-specific queries and corpora will need independent validation.
Storage flexibility and vector compression#
Both models support six vector dimension options ranging from 256 to 2,048, with three format choices: float32, int8, and binary. The storage implications compound at scale: a corpus of 100 million chunks requires roughly 819 GB as 2,048-dimensional float32 vectors, but shrinks to 102 GB with 1,024-dimensional int8, or just 3.2 GB as 256-dimensional binary vectors.
Cohere recommends 1,024-dimensional int8 for most deployments—a middle ground that cuts memory footprint substantially while maintaining near-full-precision retrieval. Binary representations offer aggressive compression suitable for initial retrieval stages prior to reranking with higher-precision models.
