Deploy Embedding Models with Hugging Face Inference Endpoints

Hugging Face · 2 years ago · 26

Hugging Face released Text Embeddings Inference (TEI), a service for deploying open-source embedding models on its Inference Endpoints platform with automatic scaling and security features. A benchmark of the BAAI/bge-base-en-v1.5 model on an Nvidia A10G instance achieved 450+ requests per second at a cost of $0.00000156 per 1,000 tokens, 64 times cheaper than OpenAI's embedding service. Developers can now deploy embedding models for retrieval-augmented generation tasks like semantic search and chatbots with minimal infrastructure management and significantly lower costs.