Blazingly fast whisper transcriptions with Inference Endpoints

Hugging Face Blog · 1 year ago

Hugging Face released an optimized OpenAI Whisper deployment for its Inference Endpoints service that achieves up to 8x faster transcription speeds using vLLM and GPU optimizations like torch.compile and float8 quantization. The optimization targets NVIDIA Ada Lovelace GPUs (L4 and L40s) and maintains transcription accuracy across standard benchmarks including the Open ASR Leaderboard datasets. Users can now deploy dedicated speech-to-text models with a single click and run inference through simple API calls.