Accelerating over 130,000 Hugging Face models with ONNX Runtime

Hugging Face Blog · 2 years ago

ONNX Runtime, a cross-platform machine learning acceleration tool, now supports over 130,000 models hosted on Hugging Face, including large language models and image generation systems. The tool achieved a 74.30% latency improvement over PyTorch when accelerating the Whisper-tiny speech model, with support covering 90 model architectures including BERT, GPT-2, and Stable Diffusion. Users can now deploy Hugging Face models with faster inference performance through ONNX Runtime integration.