Towards demystifying the creativity of diffusion models

Diffusion models have always had a slightly embarrassing secret: nobody could fully explain why they don't just regurgitate their training data. Feed one enough cat photos and, in theory, it should learn to reproduce those exact cats when you ask for noise-to-image generation. Instead it hands you a cat that looks real but has never existed. Google Research decided to dig into the actual math behind that gap, and the paper they presented at ICLR 2026 gives a surprisingly clean answer: the creativity is not mysterious at all, it is a side effect of how neural networks are trained.

The mechanism they point to is called score smoothing. Diffusion models work by learning a 'score function' that acts like a force field, pulling random noise particles toward realistic data during the denoising process. If a network learned that force field with mathematical perfection, it would carve the space into razor-sharp zones that funnel every particle straight to one of the original training points — pure memorization. But real networks trained with something like AdamW and weight decay can't produce those sharp cliffs. They round off the edges instead, and that rounding is exactly what stops particles from collapsing onto old data.

Google's team demonstrated this with almost absurdly simple experiments first: a one-dimensional world with just two training points, +1 and -1, fed into two-layer ReLU networks under varying amounts of weight decay. Crank up the regularization and the score function's midpoint transition gets gentler, so particles drift and settle somewhere between the two points instead of snapping to either one. It's a toy example, but it isolates the effect cleanly enough to build a theory around.

The more interesting part comes when they push into higher dimensions, where real images and molecules actually live, scattered thinly across what's called a data manifold inside a much larger space of pure noise. Here the smoothing effect turns out to be direction-dependent. Moving toward the manifold, the true score function is already smooth, so extra smoothing barely changes anything — good, because that's what keeps generated images sharp rather than blurry. Along the manifold itself, though, smoothing loosens the pull toward specific training points, letting the model settle into the gaps between them.

That asymmetry is really the whole story. Diffusion models aren't randomly hallucinating novelty; they're mathematically nudged into producing outputs that live in the blank spaces between real examples while still landing on the manifold that makes them look plausible. The paper's authors are careful to note this is an early result — real architectures and datasets are far messier than a two-layer ReLU network fitting two points — but it gives researchers an actual lever to pull if they want models that generalize more (or memorize less) on purpose.