Learning Montezuma’s Revenge from a single demonstration
OpenAI
OpenAI trained an AI to score 74,500 on Montezuma's Revenge using just one human playthrough as a guide. That beats every previous attempt at this famously brutal game, and it did it with a clever training trick, not brute force.
Based on reporting by OpenAI — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Montezuma's Revenge has been the go-to nightmare for reinforcement learning researchers for years. The 1984 Atari game punishes exploration: you can wander for ages without stumbling on a reward, which is exactly the kind of environment that trips up algorithms built to chase points. Most RL agents that learn from scratch barely scratch the surface of the game's early rooms.
OpenAI's fix is almost embarrassingly simple. Instead of letting the agent flail around from the very start, they hand it a single human demonstration and let it begin practice runs from specific checkpoints pulled out of that playthrough. As the agent gets good at finishing the game from a late checkpoint, the starting point creeps earlier, forcing it to master more of the game each time. Underneath all this sits PPO, Proximal Policy Optimization, the same algorithm OpenAI used to build its Dota-playing bot OpenAI Five.
The payoff is a score of 74,500, which OpenAI says tops every previously published attempt at the game. That's not just a marginal bump. Prior RL approaches to Montezuma's Revenge, even ones using curiosity-driven exploration or auxiliary rewards, have historically struggled to get an agent past the first few rooms without extensive engineering.
What's notable here isn't some exotic new architecture. It's the demonstration itself doing the heavy lifting, essentially giving the agent a curriculum instead of asking it to discover one blindly. That's a much cheaper and more general trick than most of the bespoke exploration bonuses researchers have thrown at this problem in the past, and it hints at a broader lesson: sometimes the fastest way to teach a machine isn't a smarter reward function, it's just showing it once how it's done.
My take — AI-written commentary, not fact-checked reporting
I like this because it's a reminder that the flashiest fix isn't always the smartest one — a single demo plus a well-tuned curriculum beat years of exotic curiosity-reward engineering. It also quietly undercuts the idea that harder problems always need bigger, more exotic models; sometimes they just need better use of the data you already have.
Read more about this at: OpenAI