203 summarised stories about Model Evaluation, each linking back to the original source. Browse all topics →
Simon Willison’s Weblog 2 weeks ago 14
Colin Frasier described an experiment where GPT-4o was tested on summing numbers and returning the results in words, then reran a similar test locally using Qwen3.8-27B with a DGX Spark. Qwen3.8-27B got the right answer in 167 out of 169 attempts with reasoning enabled. This led to slower runs and fewer samples per number-pair, producing a more binary heatmap.
MarkTechPost 2 weeks ago 36 ● 3 sources
OpenAI cancelled GPT-6.1 Astra on September 28 after failing internal scope and authorization tests, while GPT-6 Astra remains its top model in this lineup. GPT-6 Astra is listed at $10 per 1M input tokens and $50 per 1M output tokens, and Artificial Analysis estimates it at $4.72 per task versus Claude Fable 5.1 at $9.18 under the same list prices. The comparison shifts model-selection toward Astra for frontier tasks, Sol for high-volume coding, and Argon/Fable for specific constrained-access or cache-dependent use cases.
Exponential View 2 weeks ago 41
The newsletter reports on several items, including new claims about electricity’s pace of electrification, a Google DeepMind framework for evaluating whether AI is conscious, and OpenAI’s release of dots. Electricity now powers 46% of global GDP, while it makes up only 23% of final energy used. It reframes how far along electrification appears and updates how readers can test AI systems against consciousness criteria, while also highlighting new personal-agent and AI product developments.
The daily briefing
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.
TLDRocket
19 October 2026
The biggest thread today isn’t about model capability—it’s about who gets held responsible when AI agents act. Atlassian CEO Mike Cannon-Brookes used his company’s new “Agentic Multiplayer Protocol” to make a blunt point: “the AI did it” can’t be an escape hatch. The idea is to keep accountability with the end user by making agent edits visible in document history, like work from a colleague whose changes you can trace. It’s a practical shift away from the consumer fantasy that an agent can operate in the dark; instead, Atlassian is building a control plane where provenance and auditability are first-class features.
Read the full briefing →