Train 400x faster Static Embedding Models with Sentence Transformers
Hugging Face Blog
Researchers released training methods for static embedding models that achieve 100x to 400x faster CPU inference speeds than standard embedding models like all-mpnet-base-v2 and multilingual-e5-small. The new models retain at least 85% of the performance of their slower counterparts while using simple dictionary lookups instead of attention-based encoders. These efficiency gains enable deployment in resource-constrained environments including on-device, browser-based, and edge computing applications.
