Reptile: A scalable meta-learning algorithm

OpenAI has put out a meta-learning algorithm called Reptile, and the pitch is refreshingly modest: it does almost the same job as MAML, but without making your GPU cry.

The core trick is simple enough to explain at a whiteboard. Reptile samples a task, runs a few steps of ordinary stochastic gradient descent on it, then nudges the model's initial parameters toward wherever that training run ended up. Repeat across many tasks, and you get a starting point that's primed to adapt quickly to new, related tasks with just a handful of gradient steps. No second-order derivatives, no unrolling the training process to differentiate through it — Reptile treats the optimizer as a black box, whether that's plain SGD or Adam.

That's the real departure from MAML. MAML, the meta-learning algorithm this is compared against, computes gradients through the entire inner training loop, which means tracking second derivatives and eating extra memory and compute for the privilege. Reptile skips that step entirely and still lands close to first-order MAML in both accuracy and speed, according to OpenAI's own framing. The team ties this back to something they call the Shortest Descent algorithm, essentially applying an existing optimization idea to the meta-learning problem rather than inventing an entirely new mechanism.

What makes this worth a second look isn't raw performance — it's accessibility. A meta-learning method that only needs black-box optimizer access is one that slots into existing training pipelines without much surgery. That's the kind of unglamorous engineering choice that tends to matter more, over time, than a fractional accuracy bump.