The Sequence Knowledge - Issue 911: Distilling Diffusion and Multimodal Models

Substack · 2 months ago · 23

The Sequence Knowledge newsletter explains how distillation can be applied beyond plain next-token imitation to diffusion and multimodal models that generate via iterative editing rather than one-shot answers. Issue 911 focuses on distilling diffusion and multimodal models (compressing trajectories, distributions, motion, and semantic geometry between different worlds). This shifts distillation from copying outputs to training smaller models to reproduce the underlying generation process and cross-modal structure.