203 summarised stories about Model Evaluation, each linking back to the original source. Browse all topics →
Simon Willison’s Weblog 1 week ago 48 ● 2 sources
ttok 0.4 updated the CLI token-counting tool, fixing a Click warning and modernizing CI while adding a --list-models command using OpenAI’s tiktoken library. The release adds a --list-models option to show available models. As a result, the tool is easier to run via uvx and can display model names instead of only counting tokens.
TechCrunch 1 week ago 15 ● 19 sources
OpenAI released hundreds of claimed solutions to difficult math problems, but an advisory group says the proofs did not meet field standards focused on human-understandable results and proper formalization. Only 42% of the released proofs had not undergone the suggested formalization process. As a result, mathematicians are questioning reliance on proprietary AI models for trusted proof releases and calling for more formalization support, funding for human mathematicians, and machine-readable links between natural-language explanations and formal artifacts.
Zvi (Don't Worry About the Vase) 1 week ago 38 ● 19 sources
OpenAI released solutions to open math problems, contributing the biggest day in the history of mathematics mentioned in the article. It solved 90 of the top 500 open problems, reaching an average of three hours of Pro-level compute per question. The focus shifts to covering this math breakthrough and to ongoing updates across AI products, adoption, safety policy disputes, and new model releases.
The New Stack 1 week ago 3 ● 16 sources
Anthropic launched Claude Sonnet 5.5 shortly after Opus 5.5 and reported higher Terminal-Bench 4.0 performance plus faster output and lower token usage. Sonnet 5.5 is priced at $2 per million input tokens and $10 per million output tokens, half of Opus 5.5’s $4 and $20. Independent/API testing in the article finds Sonnet 5.5 is about 42% cheaper overall and perfect on 15 runs (Opus 13), with the main tradeoff being Opus sometimes faster on the agentic test.
MIT Technology Review 1 week ago 23 ● 3 sources
Tesla’s Optimus humanoid robot became a central focus of robotics hype, with Tesla’s CEO predicting public sales by the end of 2027 and arguing it could automate most human labor for about $20,000 per robot. Researchers pushed back that today’s language- and vision-based AI still can’t reliably handle the real world’s variability, citing the gap between making a robot that looks human and making one that moves with comparable physical skill. The debate is shifting robotics development toward evaluating approaches like robot policies trained with vision-language-action systems and alternatives such as “world model” methods to improve generality and reliability rather than relying on timelines alone.
The daily briefing
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.
TLDRocket
19 October 2026
NVIDIA’s decision to open-source Run:ai under Apache 2.0 is the day’s most consequential signal: AI compute is getting treated less like proprietary infrastructure and more like inspectable, forkable plumbing. The release puts Run:ai’s Kubernetes GPU orchestration code on GitHub and, in the local build, spins up a five-node Kind cluster pinned to Kubernetes v1.32.5 with eight simulated Tesla-A10 GPUs. That means teams can now audit scheduling behavior, adapt deployment patterns, and self-host without a paid subscription—though the community edition still stops short of NVIDIA enterprise support. In practice, it moves GPU fleet management closer to the software engineers who already live inside CI/CD pipelines, rather than keeping it behind enterprise procurement gates.
Read the full briefing →