The Sequence Knowledge - Issue #954: Learning RSI: How AI Learns to Design Learning Algorithms

Most AI systems get better at tasks without getting better at the thing underneath: learning itself. DeepMind’s DiscoRL aims straight at that machinery. It treats the update rule as something a neural network can represent, then uses a population of agents to generate the experience needed to search for a better one.

That makes this a recursive kind of problem. The system is not just training agents; it is trying to discover a learning rule that makes future agents learn more effectively. Disco57, the rule it found on Atari57, then trained fresh agents on ProcGen and DMLab-30 without any further meta-training. That is the interesting part: the rule was discovered in one setting and then carried elsewhere.

The source frames this through older ideas like MAML, learned optimizers, and Learned Policy Gradient, but DiscoRL pushes on the credit assignment problem inside the loop itself. There are really two levels to worry about: the agents learning their tasks, and the search process learning which update rule deserves to exist in the first place. That’s a messier problem than simply scaling up training.

And then there’s the practical question everyone likes to skip: what does this cost? If a learned rule only looks good after you spend a lot to discover it, the economics depend on amortizing that discovery across many later training runs. Before anyone reaches for a learned rule in production, it also needs to be interpretable enough that people can trust what it is doing.

The deeper test is harsher still. A genuinely recursive system would have to keep producing measurable gains even after compute and evaluation effort are counted properly. Otherwise it is just an expensive way to say “we tuned the optimizer.”