Build awesome datasets for video generation

Hugging Face Blog · 1 year ago

Developers have released open-source tooling for building video generation datasets, including scripts for acquiring videos, filtering by watermarks and aesthetic quality, and automatically captioning frames. The filtering pipeline demonstrated that a watermark detection threshold of 0.1 was too strict, retaining only 47 videos from 1,493 originals, while an aesthetic score threshold above 5.5 also discarded usable content. The toolkit enables communities to create custom video datasets for fine-tuning models on specific effects, as shown by datasets built for crushing and cakifying objects.