Exploring simple optimizations for SDXL

Hugging Face Blog · 2 years ago

Stability AI researchers tested optimization techniques for SDXL, their 3.5B-parameter image generation model, to reduce memory consumption and inference time. Running SDXL unoptimized required 28GB of memory and took 72.2 seconds to generate 4 images on an A100 GPU, but combining fp16 precision, scaled dot product attention, and torch.compile reduced inference time to 10.3 seconds while using 21.7GB of memory. Further memory reductions to 11.47GB are possible by adding VAE slicing and sequential CPU offloading, enabling the model to run on consumer GPUs at the cost of slower inference speed.