Accelerating SD Turbo and SDXL Turbo Inference with ONNX Runtime and Olive

Hugging Face · 2 years ago · 16

ONNX Runtime has optimized inference for SD Turbo and SDXL Turbo image generation models through CUDA and TensorRT execution providers on NVIDIA GPUs. ONNX Runtime achieved throughput gains as high as 229% for SDXL Turbo and 120% for SD Turbo compared to PyTorch across tested batch sizes and step counts. The optimizations enable faster image generation in C#, Java, and Python, with optimized model versions now available on Hugging Face.