AI Limitations

39 summarised stories about AI Limitations, each linking back to the original source. Browse all topics →

Thursday, 9 July 2026

OpenAI GPT-5.6: AI Could Do Anything, Then It Met ARC-AGI-3

The Algorithmic Bridge 6 days ago

OpenAI's GPT-5.6 Sol scored 7.8% on the ARC-AGI-3 benchmark, a test designed to measure fluid intelligence through pattern recognition games that humans solve over 90% of the time. This represents a 20-fold improvement over GPT-5.5's 0.43% score three months earlier, and the model distinguishes itself by correctly identifying game mechanics before execution rather than simply executing learned patterns. The result suggests that further progress toward general intelligence requires improved reasoning scaffolding and planning rather than raw intelligence, since the model's failures occur in multi-step inference composition rather than perception.

AI Capabilities AI Limitations Benchmark & Evaluation OpenAI

The Sequence Opinion #892: The Anatomy of a Good Environment: When Verifiability is Not Enough

TheSequence 1 week ago

The article argues that verifiability alone is insufficient for determining whether a domain is suitable for AI development, proposing instead a multi-dimensional framework where domains like mathematics and chess excel because they score highly across multiple properties including grindability. The author contrasts high-performing domains such as code and board games with struggling domains like robotics and open-ended knowledge work, suggesting the latter fail on several unstated axes despite partial strength in others. This framework explains why AI systems have made faster progress in formal domains and why some reinforcement learning environment startups may ultimately disappoint investors despite their high valuations.

AI Capabilities AI Limitations Benchmark & Evaluation Reinforcement Learning