Powerful ASR + diarization + speculative decoding with Hugging Face Inference Endpoints

Hugging Face · 2 years ago · 37

Hugging Face released a custom handler for Inference Endpoints that combines Whisper speech recognition with speaker diarization and speculative decoding into a single API. The benchmark showed speculative decoding reduced inference time from 784ms to 327ms on 8-second audio clips when using Whisper-large-v3 with distil-whisper as an assistant model. Users can now deploy modularized ASR pipelines with optional diarization and faster inference through environment variables and API configuration.