Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers

Hugging Face · 2 months ago · 42

Sentence Transformers v6.0 introduces MultiVectorEncoder, a new model type for late-interaction retrieval that keeps one vector per token instead of compressing text into a single vector, enabling stronger retrieval at the cost of larger indexes. The MaxSim operator scores queries by finding each query token's best match across all document tokens and summing those similarities, preserving both semantic understanding and exact-match capability without the lossy compression of dense embeddings. This approach trades increased storage requirements—roughly 42x more vectors per document—for improved retrieval quality on multi-requirement queries, rare entities, and out-of-domain data, with compressed indexes remaining comparable to dense embedding storage in practice.