How to Guide Your Language Flow
Apple Machine Learning Research
The authors introduce probe guidance to steer flow matching models using frozen internal states from an existing diffusion model. They evaluate it on continuous diffusion language models and report new state-of-the-art results for unconditional generation. As a result, the approach avoids an extra inference-time forward pass while aligning the weak and strong models’ dynamics.
Why it matters
We introduce a new method to guide flow matching models. Our approach, which we call probe guidance, uses the frozen internal states of an existing diffusion model to construct a guidance signal. This works using a similar principle as autoguidance, but eliminates the need for an additional forward pass at inference time and provides a reliable path to ensure that the weak and strong model share similar dynamics. We apply and benchmark this method on continuous diffusion language models, where probe guidance sets a new state-of-the-art performance on unconditional generation. When applied to a…
Related stories
Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation
Apple Machine Learning Research · 3 weeks ago ·
37
Scaling Categorical Flow Maps
Apple Machine Learning Research · 2 months ago ·
14
Consistency diffusion language models: Up to 14x faster inference without sacrificing quality
Together AI · 7 months ago ·
22