TLDRocket
Sign in

Benchmark & Evaluation

63 summarised stories about Benchmark & Evaluation, each linking back to the original source. Browse all topics →

Thursday, 9 July 2026

OpenAI GPT-5.6: AI Could Do Anything, Then It Met ARC-AGI-3

The Algorithmic Bridge 6 days ago

OpenAI's GPT-5.6 Sol scored 7.8% on the ARC-AGI-3 benchmark, a test designed to measure fluid intelligence through pattern recognition games that humans solve over 90% of the time. This represents a 20-fold improvement over GPT-5.5's 0.43% score three months earlier, and the model distinguishes itself by correctly identifying game mechanics before execution rather than simply executing learned patterns. The result suggests that further progress toward general intelligence requires improved reasoning scaffolding and planning rather than raw intelligence, since the model's failures occur in multi-step inference composition rather than perception.

Deep Learning Weekly: Issue 463

Deep Learning Weekly 1 week ago

This issue of Deep Learning Weekly covers recent AI model releases including xAI's Grok 4.5, OpenAI's GPT-Live voice model, and Mistral's open-sourced Leanstral 1.5, alongside developments in agentic AI systems and data infrastructure. Grok 4.5 achieves 80 tokens per second throughput and costs $2/$6 per million input/output tokens, while Leanstral 1.5 solves 587 out of 672 problems on PutnamBench. The newsletter highlights shifts toward agentic workloads requiring redesigned data systems, improved model interpretability through structural reasoning benchmarks, and engineering practices for optimizing AI agent efficiency and cost.

The Sequence Opinion #892: The Anatomy of a Good Environment: When Verifiability is Not Enough

TheSequence 1 week ago

The article argues that verifiability alone is insufficient for determining whether a domain is suitable for AI development, proposing instead a multi-dimensional framework where domains like mathematics and chess excel because they score highly across multiple properties including grindability. The author contrasts high-performing domains such as code and board games with struggling domains like robotics and open-ended knowledge work, suggesting the latter fail on several unstated axes despite partial strength in others. This framework explains why AI systems have made faster progress in formal domains and why some reinforcement learning environment startups may ultimately disappoint investors despite their high valuations.