Introducing Modular Diffusers - Composable Building Blocks for Diffusion Pipelines

Hugging Face just rolled out Modular Diffusers, a new way of assembling diffusion pipelines that treats each stage of the process — text encoding, image encoding, denoising, decoding — as an independent, swappable block. Rather than writing a full pipeline from scratch every time you want to tweak something, you compose pre-built or custom blocks into a workflow. It sits alongside the existing DiffusionPipeline class rather than replacing it, and the API on the surface looks nearly identical: load a model like FLUX.2 Klein 4B, call it with a prompt, get an image back.

What's different is underneath. Each block, like a text encoder or a VAE decoder, can be popped out and run on its own, or swapped for a different implementation, and the remaining blocks automatically recompose around whatever's left. Hugging Face's own example shows pulling a text-encoder block out of a Flux pipeline, running it standalone to generate prompt embeddings, then feeding those into the rest of the pipeline separately. That kind of surgery used to mean forking the whole pipeline class.

The custom blocks system is where this gets genuinely useful. Developers can write a Python class defining a block's required components, its inputs and outputs, and its computation logic, then drop it straight into someone else's pipeline. Hugging Face demonstrates this by writing a depth-map extraction block using Depth Anything V2 and inserting it at the front of a Qwen-Image ControlNet workflow — the depth block's output automatically flows downstream to wherever it's needed, no manual wiring required. Blocks like this can be published to the Hub with trust_remote_code=True, so anyone can load and reuse them, and Hugging Face has already put out a starter collection.

There's also a new