Accelerating over 130,000 Hugging Face models with ONNX Runtime

Hugging Face's model hub has become the de facto library for machine learning, and now a huge chunk of it runs faster thanks to ONNX Runtime. Microsoft's cross-platform inference engine has quietly built support for more than 90 model architectures on the hub, covering over 130,000 individual models. That's not a niche corner of the ecosystem — it includes BERT, GPT2, DistilBERT, RoBERTa, T5, Wav2Vec2, Stable Diffusion, XLM-RoBERTa, Whisper, BART, and Marian, the eleven architectures with the most uploads on the platform.

The numbers behind those architectures are staggering on their own. BERT alone accounts for roughly 28,180 models on the hub, GPT2 for about 14,060, and DistilBERT for 11,540. T5 and RoBERTa both sit above 10,000. Even Whisper, OpenAI's speech recognition model that's exploded in popularity over the past couple of years, has around 4,400 variants floating around Hugging Face. ONNX Runtime's pitch is that all of these can get a speed boost without users rewriting their inference code from scratch.

The headline example Hugging Face and Microsoft point to is whisper-tiny, where switching to ONNX Runtime cut average per-inference latency by as much as 74.30% compared to running the same model in standard PyTorch. That's the kind of gain that turns a laggy voice transcription demo into something that feels instant, and it matters more as speech and audio models get baked into consumer products where every extra millisecond is noticeable to a user.

What's notable here isn't just the performance number — it's the scale of compatibility. ONNX Runtime and Hugging Face have been coordinating directly so that whatever architecture climbs the popularity charts on the hub gets runtime support relatively quickly. That's a deliberate strategy: rather than optimizing a handful of flagship models, the goal is broad coverage across an ecosystem that's grown organically and unpredictably, driven by whatever thousands of independent developers decide to fine-tune and upload next.

For teams deploying models in production, this removes a real decision point. Instead of choosing between PyTorch's flexibility and a faster but narrower runtime, ONNX Runtime tries to make that trade-off disappear for the vast majority of architectures people actually use.