SDXL in 4 steps with Latent Consistency LoRAs

Stable Diffusion has always had a speed problem. Getting a good image out of SDXL typically means running the model through 25 to 50 denoising steps, which is fine if you're patient but miserable if you're iterating on a prompt. Latent Consistency Models tried to fix this by distilling the whole model into a faster version, but distillation is expensive: lots of data, lots of GPU time, and you have to redo it for every fine-tune you care about. Hugging Face's new approach skips that entirely.

The trick is training only a small LoRA adapter instead of the full model. That adapter, once trained on a base model like SDXL, can be dropped onto essentially any fine-tuned or dreamboothed version of that model and instantly grant it the speed of a distilled LCM — no retraining required. Swap in the LCM scheduler, set your inference steps to 4, and you're done. Hugging Face released LoRAs for SDXL 1.0, Stable Diffusion 1.5, and Segmind's SSD-1B, plus fully distilled versions of the first and last.

The numbers are the real story here. On an M1 Mac, standard SDXL takes about a minute to produce a 1024x1024 image; with the LCM LoRA it takes roughly 6 seconds. On a 4090, generation drops to under a second. Even a 10-year-old-in-GPU-years 2080 Ti goes from 10.2 seconds to 4.7. A benchmark on a lone CPU core still finished in 29 seconds versus 219. None of this required touching the base model weights.

Quality holds up surprisingly well at low step counts, though not magically so. One step gives you a vague blob. By four to six steps, images look genuinely usable, and by eight the writer found results starting to look oversaturated and cartoonish — a reminder that