Google demonstrated an RSI loop for AI discovery (paper)

Researchers tied to Google DeepMind, the University of Maryland, the University of Virginia and other groups have proposed Dream-RSI, a framework meant to make AI discovery systems better at improving themselves. The idea is not to change the underlying coding agent. Instead, a lightweight orchestration layer makes exploration explicit and programmable, then uses past discovery histories as a kind of replay simulator.

That matters because exploration is the bottleneck in recursive self-improvement. If an agent is trying to discover a better algorithm, a better math solution or a faster GPU kernel, the search can stretch across thousands of proposal-and-evaluation cycles. Fixed exploration strategies don’t learn from that history. But online policy optimization is costly, because every new strategy may need a long rollout before anyone knows if it helped.

Dream-RSI tries to sidestep that by turning completed discovery runs into reusable trees. Once a discovery history is recorded, the system can “dream” through alternative policies by replaying different branches, orders, parallel groupings and stopping choices against stored outcomes. No new executions are needed for those checks. That gives the policy developer fast feedback, then the updated policy goes back online and the loop starts again.

The paper says the method was tested on 8 scientific discovery tasks across three areas: algorithm engineering, mathematical optimization and GPU kernel engineering. In algorithm engineering, on a Lasso path solver task, Dream-RSI beat standard libraries like sklearn and strong baselines, while using up to 162 times fewer agent calls than SimpleTES and 1.7 times fewer than fixed-exploration baselines. In mathematical optimization, on sum-difference, autocorrelation and circle packing, it matched or beat strong baselines within 1k generations and saved more than 50 times the budget versus SimpleTES.

GPU work showed similar gains. On KernelBench, Dream-RSI either hit target speeds with 1.79 to 2.43 times fewer generations or improved performance by up to 2.09 times under the same budget. The broader bet is straightforward: if past discovery can be treated like a simulator, then the expensive part of self-improvement may get cheaper.