Exploring Quantization Backends in Diffusers

Hugging Face Blog · 1 year ago

Hugging Face Diffusers now supports multiple quantization backends—including bitsandbytes, torchao, Quanto, and GGUF—that compress large diffusion models like Flux while maintaining visual quality. Flux-dev at full BF16 precision requires 31.4 GB of memory, but 4-bit quantization reduces this to 12.6 GB with minimal perceptual difference in generated images. Users can now run these large models on smaller GPUs by trading some precision for dramatically reduced memory consumption and faster deployment.