Bringing Nunchaku 4-bit Diffusion Inference to Diffusers

Hugging Face · 1 month ago · 50

Hugging Face integrated Nunchaku 4-bit quantization into Diffusers, allowing diffusion models to run with 4-bit weights and activations using the SVDQuant method. A quantized text-to-image model now requires 20.6 GB of VRAM instead of 31 GB while running 1.35x faster, with torch.compile boosting that to 1.8x faster. Users can load pre-quantized models directly with from_pretrained() or quantize their own using the diffuse-compressor toolkit without custom code or local compilation.