Reptile: A scalable meta-learning algorithm
OpenAI Blog
Researchers developed Reptile, a meta-learning algorithm that samples tasks, applies stochastic gradient descent to each, and updates initial parameters based on the learned results. Reptile achieves computational efficiency comparable to first-order MAML while requiring only black-box optimizer access. The algorithm enables faster adaptation to new tasks by learning better initialization points across multiple learning episodes.
Why it matters
We’ve developed a simple meta-learning algorithm called Reptile which works by repeatedly sampling a task, performing stochastic gradient descent on it, and updating the initial parameters towards the final parameters learned on that task. Reptile is the application of the Shortest Descent algorithm to the meta-learning setting, and is mathematically similar to first-order MAML (which is a version of the well-known MAML algorithm) that only needs black-box access to an optimizer such as SGD or Adam, with similar computational efficiency and performance.
