Limits of Confidence in Diffusion

Apple Machine Learning Research · 1 week ago · 25

The paper shows that discrete diffusion steps only match the training distribution when the positions written are conditionally independent given the already-fixed tokens, and that products of per-position distributions cannot reproduce dependent token groups. On the ScanAndAdd task, it measures the generated distribution at 29× the sampling-noise floor total variation while per-sample metrics are 1.0. This limits how confidence rankings that write per-position distributions can correct dependence errors, even when the task’s joint distribution is known.