A picture's worth a thousand (private) words: Hierarchical generation of coherent synthetic photo albums

Google Research · 8 months ago

Researchers at Google developed a method to generate synthetic photo albums that maintain privacy through differential privacy while preserving thematic coherence across images. The approach uses a hierarchical text-based intermediary process where photos are first converted to captions and album summaries, these representations are privately fine-tuned using large language models, and then converted back to images via text-to-image generation. The method achieved high semantic similarity (measured by MAUVE scores) between real and synthetic albums when tested on the YFCC100M dataset of nearly 100 million Creative Commons images.