Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models

Apple Machine Learning Research · 3 weeks ago · 35

Researchers compared the performance characteristics of diffusion language models (DLMs) and autoregressive language models (ARMs) across inference scenarios. DLMs achieve higher arithmetic intensity through parallel token generation but fail to scale effectively with longer contexts, while ARMs maintain superior throughput in batched inference. The key finding is that reducing sampling steps in DLMs is essential for them to achieve lower latency than ARMs in practical deployments.