Consistency diffusion language models: Up to 14x faster inference without sacrificing quality

Together AI · 4 months ago

Researchers introduced Consistency Diffusion Language Models (CDLM), which accelerates diffusion-based language model inference by combining consistency-based training with block-wise key-value caching. The method reduces refinement steps by 4.1x to 7.7x and achieves latency improvements up to 14.5x on coding benchmarks while maintaining quality through trajectory-consistent training objectives. This enables diffusion models to compete with autoregressive approaches on inference speed while preserving their bidirectional context capabilities.