Designing synthetic datasets for the real world: Mechanism design and reasoning from first principles

Every AI lab has hit the same wall eventually: the internet is finite, and for a lot of the stuff that actually matters — rare cyberattacks, obscure legal precedents, privacy-sensitive medical scenarios — there just isn't enough real data lying around to train on. Google Research's answer is a system called Simula, detailed in a new paper in Transactions on Machine Learning Research, and its pitch is refreshingly specific: stop treating synthetic data generation as a black box and start treating it as mechanism design.

Most synthetic data pipelines today lean on manual prompting, evolutionary tweaking, or a pile of seed examples pulled from the target domain. Davidson, Seguin, Bacis, Ilharco and Harkous — the team behind the paper — argue that approach caps out fast. It's hard to scale, hard to explain, and it optimizes one data point at a time instead of shaping a dataset as a whole. Simula flips that by using reasoning models to build deep hierarchical taxonomies of a domain from scratch, no seed data required, then treating diversity, difficulty and correctness as separate dials you can turn independently.

The process runs in four stages. First, global diversification maps out the conceptual long tail of a domain so the system isn't just clustering around the obvious, common cases. Then local diversification generates multiple distinct takes on the same underlying scenario — so