PaperBench: Evaluating AI’s Ability to Replicate AI Research

OpenAI's newest benchmark doesn't ask a model to answer trivia or pass a coding puzzle. It asks an AI agent to take a real, recent machine learning paper and rebuild it: read the paper, write the code, run the experiments, and get results that match what the original researchers reported. That's PaperBench, and it's built from 20 papers pulled from ICML 2024's spotlight and oral tracks, spanning areas like reinforcement learning and language model architectures.

The grading isn't a vague vibe check either. OpenAI worked with the original paper authors to build detailed rubrics, breaking each replication attempt into more than 8,000 gradable subtasks. Did the agent implement the right algorithm? Did it produce a chart that lines up with the paper's reported numbers? Every piece gets scored individually, which is a lot more rigorous than just asking a model 'did you succeed, yes or no.'

And here's the part that should temper some of the AGI-is-imminent chatter: the best-performing agent setup, built on Claude 3.5 Sonnet with a custom scaffold, averaged only around 21% replication score across the 20 papers. Human machine learning PhDs, given 48 hours to attempt the same task, hit roughly 41%. Twice as good, working under a hard time limit no less. That gap is the real headline here, not the fact that AI attempted this at all.

OpenAI also had to solve a smaller but tricky problem: how do you grade thousands of these submissions without hiring an army of PhDs for every future benchmark run. Their answer was an LLM-based judge, which they validated by comparing its scores against human graders and finding solid agreement. That judge, released alongside PaperBench, might end up being just as useful to other researchers as the benchmark itself, since automated grading of open-ended technical work is its own unsolved headache.

What PaperBench really measures is something narrower and more useful than 'can AI do research.' It measures whether an agent can take dense, ambiguous scientific writing and turn it into working, verifiable code under real constraints. Right now, the answer is: sort of, partially, and not nearly as well as a stressed-out grad student on a deadline.