360 summarised stories about AI Evaluation, each linking back to the original source. Browse all topics →
YouTube 3 weeks ago 34 ● 3 sources
Corey and Grant stress-tested GPT-6 Astra with six one-shot build tasks and minimal follow-up steering. One of the tasks was a black hole simulator. The results are that the model can still produce working builds across varied prototypes—from Blender and a physics game to a sci-fi world, sound diagnosis, and Cat Doom.
TechCrunch 3 weeks ago 51 ● 3 sources
OpenAI formed an independent advisory group on mathematics and artificial intelligence hosted by the Institute for Advanced Study in Princeton to provide mathematicians input on its math research. OpenAI said its internal model has resolved more than 100 additional open problems. The group will offer advisory and release-coordination input without decision-making power or authority to change OpenAI’s pace.
The New Stack 3 weeks ago 20
Grok Build and Claude Code were tested to see whether each terminal coding agent remembered instructions after you closed and reopened it, with results tracked across three scenarios. Claude Code used 576,863 tokens and $1.05 total versus Grok Build’s 390,848 tokens and $0.41. Grok matched on two tests and passed all three by carrying cross-project rules via a global memory scope, while Claude Code failed the cross-project test and was slower and more expensive overall.
The New Stack 3 weeks ago 17 ● 9 sources
SpaceXAI released Grok 4.7, a coding agent model trained to handle long, hours-long runs by focusing reinforcement learning on harder tasks and improving self-verification and long-context management. Grok 4.7 scored 38.0% on Terminal-Bench 4.0, up from 20.3% for Grok 4.6. The model’s long-horizon performance improved substantially, but it still fails most of the time in long-running codebase evaluations, so reliability remains a limiting factor.
The New Stack 3 weeks ago 15 ● 3 sources
AWS open-sourced Strands Harness, a general-purpose AI agent built on its Strands Agents SDK that can run locally or be deployed to various clouds. AWS says it was 45% cheaper than Claude Code and Codex in its benchmark comparison. Developers can now start from a preconfigured agent loop with built-in context and session handling while still swapping model providers and customizing tools and storage.
Amazon Science 3 weeks ago 31
Amazon Bio Discovery’s research team published three papers introducing AI methods to better predict binding and developability properties and to design antibodies or nanobodies for experimentally validated candidates. MochiBind scored 200,000 antibody pairs in roughly 13 seconds on a CPU, reporting more than a 100-fold inference speedup over competing methods. The work shifts antibody evaluation toward cross-antigen relative binding and batch-effect-aware context prediction, and it combines an epitope hotspot agent with de novo generative design to produce lab-validated hits against a novel cancer target.
Hugging Face 3 weeks ago 48
Researchers reformulated transformer block pruning as a constrained binary optimization mapped to an Ising glass, enabling fast scoring of block-removal combinations without running full benchmarks. At 50% depth pruning of Llama-3.3-70B-Instruct (removing 40 of 80 blocks), it reported an almost 23-point MMLU advantage over the best block-removal baseline. The method shifts block selection from independent/heuristic ranking to searching low-energy coupled configurations, including “excited” low-energy states that can outperform the single best solution.
OpenAI 3 weeks ago 23 ● 3 sources
OpenAI is working with an independent Advisory Group on Mathematics and Artificial Intelligence to guide how emerging AI results are reviewed and communicated.
Boris Cherny’s Blog 3 weeks ago 38
The author describes an iterative process for product building that repeatedly redefines the problem and approach when new information appears, acknowledging they are often wrong during execution. The note says the process is implemented “many tries” for complicated problems. This changes teams’ expectations toward real-time feedback, clearer problem/approach definition, and updating assumptions as new data arrives.
danluu.com 3 weeks ago 27
The article argues that people using LLMs while “turning off” their own judgment—treating outputs as if they automatically work—often get wrong or unusable results. It says that being a “meat proxy” only sort of works as of September 2026. As a result, the author concludes that high-value work still needs human QA/architecture and active supervision to handle out-of-distribution decisions and avoid issues like overfitting to tests or metrics.
OpenAI 3 weeks ago 8 ● 2 sources
OpenAI outlined a path toward shared global AI standards focused on coordinated evaluation, reporting, and governance for safety.
The daily briefing
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.
TLDRocket
19 October 2026
NVIDIA’s decision to open-source Run:ai under Apache 2.0 is the day’s most consequential signal: AI compute is getting treated less like proprietary infrastructure and more like inspectable, forkable plumbing. The release puts Run:ai’s Kubernetes GPU orchestration code on GitHub and, in the local build, spins up a five-node Kind cluster pinned to Kubernetes v1.32.5 with eight simulated Tesla-A10 GPUs. That means teams can now audit scheduling behavior, adapt deployment patterns, and self-host without a paid subscription—though the community edition still stops short of NVIDIA enterprise support. In practice, it moves GPU fleet management closer to the software engineers who already live inside CI/CD pipelines, rather than keeping it behind enterprise procurement gates.
Read the full briefing →