A picture's worth a thousand (private) words: Hierarchical generation of coherent synthetic photo albums

Google Research just tackled a problem that sounds niche but actually matters a lot for anyone building AI on personal photo data: how do you create a fake version of someone's photo album that's both useless for identifying them and still coherent enough to be useful for training models?

The trick is sneaky and a little poetic. Instead of trying to privately generate images directly, the team converts each photo album into text first. An AI model captions every photo, then summarizes the whole album, turning a sequence of images into a structured block of language. Two separate language models are then fine-tuned with differential privacy — one to write album summaries, one to write photo captions based on those summaries — and the whole thing runs in reverse to spit out new albums: generate a summary, then generate captions from it, then hand those captions to a text-to-image model to produce actual pictures.

Why bother with text as a middleman? Because language models are simply better at generating coherent text than diffusion models are at generating coherent sequences of images, and because text is a lossy compression of an image by nature — so even without formal privacy guarantees, describing a photo in words already scrubs out a lot of identifying detail. It's also cheaper. Generating text is far less computationally expensive than generating images, so Google's approach filters out irrelevant albums using cheap text before spending money on the costly image-generation step.

The hierarchical structure — summary first, then captions — solves a second problem: keeping multiple photos in an album thematically consistent with each other. Since every caption in an album is generated using the same summary as context, the resulting synthetic photos actually look like they belong together, rather than being a random grab-bag of unrelated images. It also happens to be cheaper to train, since self-attention costs scale quadratically with context length, so splitting the job into two shorter-context models beats training one model on long combined sequences.

Google tested this on YFCC100M, a public dataset of roughly 100 million Creative Commons photos, grouping images into albums based on which user took them within the same hour. Using MAUVE scores to measure semantic similarity, and comparing topic distributions between real and synthetic album summaries, the team found the synthetic albums closely tracked the real ones, both in content and visual theme, after differential privacy fine-tuning was applied.