The Design Space of Tri-Modal Masked Diffusion Models

Apple Machine Learning Research · 1 day ago · 9

Tri-modal Masked Diffusion Models were introduced, pretrained from scratch on text, image-text, and audio-text data to enable generation across three modalities. The work presents a batch-size analysis linked to a novel stochastic differential equation (SDE) based method. It changes how these diffusion models are scaled and sampled, by adding guidance on mixing ratios, noise schedules, batch effects, and optimized inference defaults.