Introducing the FFASR Leaderboard: Benchmarking ASR in the Real World

Hugging Face · 2 months ago · 28

Treble Technologies and Hugging Face launched the FFASR Leaderboard, an open benchmark for evaluating automatic speech recognition models under realistic far-field acoustic conditions including reverberation, background noise, and varying microphone distances. The benchmark tests models across 14 simulated rooms at three signal-to-noise ratios, with performance measured against an 8-hour held-out test set, while also reporting inference speed (RTFx) on identical NVIDIA L4 GPU hardware. The leaderboard reveals that far-field word error rates at low signal-to-noise ratio are consistently several times higher than near-field performance on the same speech content, providing visibility into the gap between clean-speech benchmarks and real-world deployment that was previously difficult to measure.