TLDRocket
Sign in

AI Benchmarking

294 summarised stories about AI Benchmarking, each linking back to the original source. Browse all topics →

+ Follow this topic

Wednesday, 12 August 2026

MindTopo reveals VLMs’ spatial reasoning abilities

Microsoft 2 months ago 22

MindTopo, a new benchmark from Microsoft Research, evaluates whether multimodal large language models can do 3D topological reasoning and planning, not just recognize topology in single images. It tests topology across 5 categories: continuity, separation, order, enclosure, and knots, with both reasoning and planning tasks. Results show a persistent gap where models perform better on static recognition than on interactive action sequences that must preserve or change the same topological relations.

QOBLIB: tracking progress in quantum optimization

IBM Research 2 months ago 47

QOBLIB was established as a quantum optimization benchmarking resource through a Nature Computational Science publication and a new website that organizes and tracks comparable results. The library reports 2,000+ submitted results and 500+ instances already solved to optimality. By making benchmarks and submissions more accessible, QOBLIB is increasing community participation and raising the bar for future “quantum advantage” claims against updated classical baselines.

Empty shelves or lost keys? Recall is the bottleneck for parametric factuality

Google Research 2 months ago 47

Knowledge profiling found that many factual mistakes in frontier LLMs are recall failures (encoded facts not reliably accessible) rather than encoding failures (facts not encoded). 95–98% of facts were encoded in Gemini-3-Pro and GPT-5, yet direct recall failed on 26–34% of facts. The implication is a shift toward interventions that improve knowledge utilization (including thinking/recovery) instead of primarily scaling to add missing knowledge.

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.