Towards demystifying the creativity of diffusion models

Google Research · 21 hours ago

Researchers studied how diffusion models generate novel images and data beyond their training examples, finding that the models' creativity stems from neural network regularization effects that blur the learned score function. In a one-dimensional example with two training points at +1 and -1, weight decay causes the neural network to learn a smoother approximation of the score function, allowing generated particles to settle between the training points rather than copying them exactly. This score smoothing mechanism enables diffusion models to balance realism with novelty by interpolating between training data points while maintaining the quality of high-dimensional data manifolds.