The engineer teaching AI the difference between real and fake
Tech Funding News 2 months ago 34
Mecka AI founder Saurab Dhir builds systems that decide which crowdsourced recordings of human physical work are real rather than faked.
Model Evaluation
206 summarised stories about Model Evaluation, each linking back to the original source. Browse all topics →
Key players
Wednesday, 12 August 2026
MindTopo reveals VLMs’ spatial reasoning abilities
Microsoft 2 months ago 22
MindTopo, a new benchmark from Microsoft Research, evaluates whether multimodal large language models can do 3D topological reasoning and planning, not just recognize topology in single images. It tests topology across 5 categories: continuity, separation, order, enclosure, and knots, with both reasoning and planning tasks. Results show a persistent gap where models perform better on static recognition than on interactive action sequences that must preserve or change the same topological relations.
Inside the Data Bottleneck Slowing Visual and Physical AI
Wiley Science and Engineering Content Hub 2 months ago 7
A white paper from IEEE Spectrum and Wiley reports how more than 700 practitioners build visual and physical AI and why models fail. It says 78% of teams already see measurable value from visual and physical AI while 74% view the field as underinvested. The report argues that successful teams spend nearly 3x more time on data work and that data curation—not larger model architectures or collecting more data—drives production outcomes.
I wrote an AI textbook — how long until AI can do it better?
Interconnects 2 months ago 9
The author argues that today’s LLMs have stalled on long-form non-fiction and technical writing, even as they improved on tasks like coding and math. They report running GPT 5.5 Pro on a near-final, 200-300 page manuscript and it found deep, surprising minor typos. The result, they say, is that authors will keep using AI more for editing and small sections (not full chapters), while humans provide the organization and insight until models can better revisit and string ideas together.
QOBLIB: tracking progress in quantum optimization
IBM Research 2 months ago 47
QOBLIB was established as a quantum optimization benchmarking resource through a Nature Computational Science publication and a new website that organizes and tracks comparable results. The library reports 2,000+ submitted results and 500+ instances already solved to optimality. By making benchmarks and submissions more accessible, QOBLIB is increasing community participation and raising the bar for future “quantum advantage” claims against updated classical baselines.
Launch HN: Discovered Materials (YC P26) – AI agents to discover new materials
discoveredmaterials.com 2 months ago 38 ● 2 sources
Discovered Materials launched an HN post describing AI agents that computationally discover new, dynamically stable semiconductor materials. The post cites model testing over an 8 hour run that produced results that would take a PhD student a couple of weeks. It shifts expectations by providing a benchmark and releasing hundreds of newly discovered materials while arguing that future impact will depend on adding synthesis and lab validation, where they say current models are weaker.