🧨 Accelerating Stable Diffusion XL Inference with JAX on Cloud TPU v5e
Hugging Face
Hugging Face's Diffusers now runs Stable Diffusion XL on Google's Cloud TPU v5e using JAX, and it's fast. Four 1024x1024 images in about 2 seconds once compiled, at nearly half the cost of the previous TPU generation.
Based on reporting by Hugging Face — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
SDXL is a beast compared to its predecessor — the UNet alone is roughly three times bigger — which means the usual memory and latency headaches got worse, not better. Hugging Face's answer is to lean hard into JAX and Google's newest TPU silicon, and the numbers they're showing off are hard to ignore.
The trick is combining two things JAX does well. First, just-in-time compilation locks in fixed input and output shapes, then produces a highly optimized TPU binary that gets reused on every subsequent call. Image generation happens to be a near-perfect fit for this, since prompt embeddings and image sizes stay constant across requests. Second, JAX's pmap lets that compiled function fan out across multiple TPU chips at once — an 8-chip setup cranks out 8 images in the time a single chip needs for one.
Hugging Face published a full walkthrough, and it's refreshingly short: load FlaxStableDiffusionXLPipeline, cast weights to bfloat16 (but keep the scheduler state in float32 or you'll get black images — a real gotcha they call out explicitly), replicate params and prompts across devices, then call the pipeline with jit=True. The first call eats about three minutes tracing and compiling. Every call after that is the fast one.
And fast is the right word. On a TPU v5e-4, four 1024×1024 images render in roughly 2.3 seconds of actual generation time, with the full round trip — formatting, networking, frontend — landing around 4 seconds in their live demo. Benchmarked against TPU v4-8 at 20 steps, v5e-4 hits about 21.5 performance-per-dollar versus 9.05 for v4-8 at batch size 4, which works out to roughly 2.4x better cost-efficiency. TPU v5e itself is priced at less than half of v4, so the perf/$ gap isn't just about raw speed — it's about Google undercutting its own previous generation on price while still gaining throughput.
The demo running behind this post is deliberately unglamorous: a handful of pre-allocated TPU v5e-4 backends behind a random load balancer. Hugging Face says a GKE-based autoscaling version is coming later. For now, the code is open source in Diffusers, which matters more than the demo itself — anyone running SDXL at scale on TPUs gets a tested recipe instead of having to reverse-engineer JAX's shape-static quirks themselves.
My take — AI-written commentary, not fact-checked reporting
This is a genuinely useful engineering post, not hype dressed up as one — the perf/$ numbers are concrete and the bfloat16-vs-float32 scheduler gotcha alone will save someone a debugging afternoon. My only gripe is that this is Hugging Face marketing Google's hardware roadmap as much as its own library, and that's fine, but it's worth remembering TPU v5e only helps you if you're already committed to Google Cloud — the real story here is JAX's compilation model being a great fit for diffusion, and that insight travels well beyond TPUs.
Read more about this at: Hugging Face