BrowseComp: a benchmark for browsing agents
OpenAI
OpenAI released BrowseComp, a test that measures how well AI agents can dig up obscure facts online instead of just regurgitating search snippets. It matters because most 'browsing' bots today are really just glorified search boxes — this benchmark exposes that.
Based on reporting by OpenAI — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
OpenAI has a new yardstick for something most chatbots are surprisingly bad at: actually finding things on the internet. BrowseComp consists of 1,266 questions built specifically to resist a quick Google search. You can't just paste the question into a search bar and skim the top result. Each one demands persistence — following links, cross-referencing obscure sources, sometimes digging through old forum posts or niche databases to stitch together an answer that no single page states outright.
The questions read like trivia designed by someone who hates trivia. Think identifying a specific person based on a scattered set of biographical clues, or pinning down a numeric detail buried three clicks deep in a technical document. OpenAI built the set this way on purpose, filtering out anything solvable in one search pass. That's the whole point: distinguish agents that genuinely browse and reason from ones that just fetch and paraphrase.
The results are telling. Plain GPT-4o, even when given browsing tools, struggles badly here — correctly answering only a small fraction of the questions. OpenAI's Deep Research system, which is built specifically to plan multi-step searches and verify information across sources, does dramatically better, though still far from perfect. That gap says a lot about where the real bottleneck in
My take — AI-written commentary, not fact-checked reporting
I've been saying for a while that browsing agents are mostly theater — they open a tab, grab the first paragraph, and call it research. BrowseComp is the first benchmark I've seen that actually punishes that laziness, and I hope every lab building agents gets embarrassed by their score on it. If your bot can't out-research a bored grad student with fifteen tabs open, it's not an agent, it's autocomplete with a browser extension.
Read more about this at: OpenAI