TLDRocket
Sign in

AI Evaluation

360 summarised stories about AI Evaluation, each linking back to the original source. Browse all topics →

+ Follow this topic

Tuesday, 22 September 2026

GPT-6 Sol closes most of the alignment gap with Astra. It’s one-fifth the price.

The New Stack 3 weeks ago 23 ● 8 sources

OpenAI released GPT-6 Sol and GPT-6 Luna as expanded GPT-6 models, targeting Astra’s higher alignment with lower cost. GPT-6 Sol costs $2 per million input tokens versus $10 for GPT-6 Astra. OpenAI reports much lower alignment gaps on internal evaluations (e.g., deception and unauthorized actions) while noting uncertainty remains about whether monitoring and observability limitations will also carry over.

OpenAI releases GPT-6 Sol and Luna — and cuts token prices in half

The New Stack 3 weeks ago 33 ● 8 sources

OpenAI released GPT-6 Sol and GPT-6 Luna to complement its flagship GPT-6 Astra, with no GPT-6 Terra in the lineup. GPT-6 Sol and GPT-6 Luna cut default prices to $2/$10 and $0.10/$0.50 per million input/output tokens, respectively (down from GPT-5.6 Sol’s $4/$20 and $0.20/$1.20). OpenAI’s changes mean lower token costs, improved prompt caching with up to 90% discounts for cached input, and availability in ChatGPT Work and Codex for paid tiers starting Tuesday (with a gradual rollout).

Claude Opus 5.5: Anthropic Launches New Top Model Despite Calling for AI Slowdown

Trending Topics 3 weeks ago 3 ● 3 sources

Anthropic released Claude Opus 5.5, its first model in the Claude 5.5 family, despite days earlier arguing for AI development to slow down. Artificial Analysis measured Opus 5.5 scoring 58 at maximum effort on its Intelligence Index, the highest result it had measured. The launch pairs that top ranking with lower token prices and added API restrictions, while leaving some benchmarks where competitors still lead.

Evaluate skill-equipped agents with Strands Evals and Amazon Bedrock AgentCore

Amazon Web Services 3 weeks ago 37

Strands Evals SDK and Amazon Bedrock AgentCore Evaluations added skill-focused evaluators to measure whether skill-equipped agents choose the right skill and follow its prescribed steps. The Skill Instruction Following evaluator rates each invoked skill on a five-level scale and uses evidence to detect skipped steps. Teams can use the per-skill results from recorded trajectories or OpenTelemetry traces to diagnose whether failures come from routing (wrong skill) or execution (partial instruction following) and turn those checks into deployment gates for regressions.

Teradata expands AI assistant with tools for governed agents

SiliconANGLE 3 weeks ago 15

Teradata expanded its Tera AI assistant with a context engine and execution system aimed at helping governed data agents carry out enterprise tasks. In company testing on SWE-bench Pro with the same Opus 5 model, Tera used 73% fewer tokens and finished tasks 42% faster than Claude Code. The update adds traced, policy-aware context for agent requests, tool execution with human approval controls, reusable agent skills, and new capabilities scheduled for availability in Q4 2026.

People Training OpenAI’s AI Fired for Using AI to Train the AI

404 Media 3 weeks ago 8 ● 23 sources

Contractors hired to improve OpenAI’s models were fired after using AI to help complete their review work. One contractor described being removed for “authenticity” issues tied to AI-assisted work. OpenAI’s training pipeline becomes less reliable and more prone to the broader risk of model degradation as the people tasked with preventing it rely on the same tools.

The Sequence Knowledge - Issue 937: RSI in Post-Training: The Loop That Already Shipped

TheSequence 3 weeks ago 16 ● 2 sources

The article describes the post-training self-improvement loop where a model generates candidate answers, a grader selects the good ones for training data, and the model retrains to produce better answers. It says this loop has been run at industrial scale for two years across frontier labs. As a result, the focus shifts from speculative self-modifying agents to the already-deployed post-training pipeline as the main driver of model improvements.

Jev: A New AI Model That Refuses to Write Text Has Developers Hooked

Trending Topics 3 weeks ago 43 ● 19 sources

TypeSafe AI released the Jev AI model, which generates structured outputs from a predefined schema rather than writing text, and its API briefly stopped serving requests due to high developer demand. Jev reports end-to-end response times of 70 to 500 milliseconds and charges $0.042 per million input tokens with free output tokens. Developers are shifting toward schema-based “model routing” and other agent/tool workflows for faster, more type-safe automation, while independent verification and auditability of confidence and behavior remain open.

Grok 4.7 Pushes SpaceXAI Into the Top 4 of AI Labs

Trending Topics 3 weeks ago 45 ● 9 sources

SpaceXAI released Grok 4.7, moving the company’s model into the top group of frontier labs on the Artificial Analysis Intelligence Index. Grok 4.7 scored 46 points, up 2 points from Grok 4.6, placing it fourth using the xhigh reasoning setting. The update improves agentic knowledge work and coding-agent benchmarks but increases compute to about 81,000 output tokens per task and rolls out a new safety stack and invite-only red-team access.

Canary rollouts: upgrade models in production without downtime

Together AI 3 weeks ago 4

Rollouts automate traffic migration between two model deployments by running canary, blue-green, or rolling upgrades with built-in health checks and rollback controls. The default canary traffic ladder is 5% → 25% → 50% → 100% with a wait window of at least 390 seconds per step when using metric gates. This replaces manual hard swaps or DIY staged cutovers by letting the platform enforce step ordering, pause on failed gates, and complete or cancel without leaving traffic in an ambiguous state.

How UK AISI and EvalEval Are Making Benchmark Results Reproducible

Hugging Face 3 weeks ago 10

EvalEval and the UK AI Security Institute (AISI) have expanded their collaboration by using EvalEval’s Evaluation Cards infrastructure to publicly share reproducible benchmark evaluation methods and results. The release includes verified results for five benchmarks from AISI’s main experiment and covers six frontier models, including GPT-5.4. This makes the evaluation setup and configurations available alongside results for those benchmarks, enabling closer comparison and reproducibility across studies.

Priorities and principles for effective third party assessments

OpenAI 3 weeks ago 10 ● 2 sources

OpenAI outlined priorities and principles for rigorous, secure, independent third-party assessments of frontier AI models and safety safeguards. The article calls for these assessments to cover both frontier models and the safeguards themselves. This sets expectations for how external reviewers should evaluate AI safety going forward.

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.