Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
Apple Machine Learning Research
Researchers compared the performance characteristics of diffusion language models (DLMs) and autoregressive language models (ARMs) across inference scenarios. DLMs achieve higher arithmetic intensity through parallel token generation but fail to scale effectively with longer contexts, while ARMs maintain superior throughput in batched inference. The key finding is that reducing sampling steps in DLMs is essential for them to achieve lower latency than ARMs in practical deployments.
Why it matters
Large Language Models (LLMs) have achieved state-of-the-art performance on a broad range of Natural Language Processing (NLP) tasks, including document processing and code generation. Autoregressive Language Models (ARMs), which generate tokens sequentially conditioned on all previous tokens, have been the predominant paradigm for LLMs. While these models have achieved high accuracy across a range of downstream tasks, they exhibit low arithmetic intensity due to the inherent sequential dependency in next-token prediction. Recently, Diffusion Language Models (DLMs) have emerged as a promising…