Train 400x faster Static Embedding Models with Sentence Transformers

Hugging Face · 1 year ago · 41

Researchers released training methods for static embedding models that achieve 100x to 400x faster CPU inference speeds than standard embedding models like all-mpnet-base-v2 and multilingual-e5-small. The new models retain at least 85% of the performance of their slower counterparts while using simple dictionary lookups instead of attention-based encoders. These efficiency gains enable deployment in resource-constrained environments including on-device, browser-based, and edge computing applications.