Diffusion Models for Video Generation

Lilian Weng · 2 years ago

Researchers are extending diffusion models from image synthesis to video generation, a more complex task requiring temporal consistency across frames. Video generation demands larger high-quality datasets and text-video pairs, which are harder to obtain than image datasets. This capability enables models to generate coherent multi-frame sequences rather than single static images.