TLDRocket
Sign in

AI Evaluation

358 summarised stories about AI Evaluation, each linking back to the original source. Browse all topics →

+ Follow this topic

Wednesday, 7 October 2026

The Curve Bends You

Zvi (Don't Worry About the Vase) 1 week ago 50

The Curve’s third season brought together people to debate AI alignment and pacing while expressing concern that labs are moving faster than they can implement safer alignment plans. Situational awareness was reported down by 0.41 points on a 10-point scale despite rapid AI progress in the prior year. As a result, attendees emphasised the need to slow execution of “automated alignment” and increase coordination between DC, the public, and labs under Chatham House rules.

Does better work always mean better workers?

Google Research 1 week ago 48

A field experiment tested whether a Google AI patent-writing assistant improves patent lawyers’ performance and durable judgment when the tool is available versus removed after three months. After 10 days, AI access increased drafting scores by 0.34 standard deviations (about a 10-point percentile gain) and the gap remained similar at 90 days (0.38 SD). The short-term drafting boost did not translate to improved unassisted redlining for junior lawyers, so AI helped immediate output but did not reliably build long-term expertise.

OpenAI’s former safety lead says it’s shipping ‘new capability and risk every Tuesday’

The New Stack 1 week ago 25

David Robinson, OpenAI’s former safety lead, said OpenAI’s releases now combine faster updates to reasoning and tools rather than waiting for full retraining runs. The release rhythm he cited is “every Tuesday,” when OpenAI ships new capability and risk. As a result, safety documentation and testing cadence are strained, with safety evaluations needing to cover changes that occur between major model releases rather than only at new frontier-model launches.

Agent Lightning v1.0: A 3,500-Line Lightweight Agentic RL Framework for Training Agents with Real Harnesses

Microsoft 1 week ago 13

Microsoft Research Asia introduced Harnessed Agentic RL by open-sourcing Agent Lightning v1.0, where the same agent harness used in deployment participates directly in reinforcement learning without being reimplemented in the training loop. The paper reports raising Qwen3.5-9B Pass@1 on SWE-bench Verified from 41.8% to 56.4% using about 6,000 training samples. Agent training shifts from tokenizing a rebuilt interaction loop to observing real harness model calls via a proxy, enabling end-to-end RL control on standard Kubernetes jobs with a lightweight ~3,500-line codebase.

GPT-6.1 Sol vs. GPT-6 Astra: Same accuracy at 18% of the cost

The New Stack 1 week ago 49 ● 3 sources

OpenAI launched GPT-6.1 Sol and it was tested against GPT-6 Astra on three identical agentic coding and log/decoder tasks. GPT-6.1 Sol completed 15 runs in $2.66 total versus $14.77 for Astra, which is 18% of the cost. The tests found the models tied on accuracy across all runs, while GPT-6.1 Sol was faster on the longer incident logs and resolver tasks.

Kore.ai launches Autoloop to keep tuning enterprise AI agents after they go live

SiliconANGLE 1 week ago 13

Kore.ai launched Autoloop, an optimization engine that continuously tunes enterprise AI agents after they go live on its Agent Platform. Autoloop cites its 2026 Agent Productivity Index showing 79% of enterprises had reversed an action taken by an AI agent. This adds automated, goal-based iteration using trace-based validation and targeted blueprint-based fixes instead of manual one-off patching.

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.