EvoLib: Turning experience into evolving knowledge

Every AI agent that remembers things eventually runs into the same problem: memory turns into clutter. Store enough conversations, code attempts, and reasoning traces, and you end up with a pile of experience that's technically searchable but practically useless. Microsoft Research thinks it has a fix, and it's called EvoLib.

The idea borrows from something humans do without thinking. Nobody replays every detail of a bad decision from three years ago. Instead, we keep the lesson and drop the noise. EvoLib tries to do the same thing for language models, converting raw attempts into two kinds of distilled knowledge: reusable skills pulled from things that worked, and reflective insights pulled from things that didn't. Crucially, none of this requires touching the model's weights, so it works on closed, API-only systems just as well as open ones.

What makes EvoLib more than a fancy memory cache is what happens after knowledge gets stored. The system actively consolidates similar entries into more general versions, so a trick learned on one math problem can eventually generalize to a whole class of problems. It also runs a weighting scheme that tracks not just how useful a piece of knowledge was on the task at hand, but how much it helps future tasks — meaning the library reshuffles itself over time, promoting the entries with real staying power and letting the rest fade.

Microsoft tested this across math reasoning, coding under efficiency constraints, and long-horizon decision-making tasks, and reported that EvoLib beat both retrieval-based memory systems and other abstract-memory approaches, while using fewer tokens to do it. More interesting is the compute-scaling result: as you throw more test-time compute at the problem, EvoLib turns that compute into performance gains faster than rivals that treat each task in isolation. The team also shuffled the order tasks arrived in and found performance stayed steady, which matters a lot if you imagine a real deployed agent getting hit with a random mix of user requests rather than a tidy curriculum.

The bigger claim buried in here is a bet on where AI progress goes next. Instead of just scaling model size or throwing more compute at inference, Microsoft is arguing that the compounding value comes from letting systems keep and refine what they've learned after deployment. Code and results are already public on GitHub, so outside researchers can poke at whether the gains hold up beyond the paper's chosen benchmarks.