Model Evaluation

205 summarised stories about Model Evaluation, each linking back to the original source. Browse all topics →

Key players

OpenAI Anthropic GPT-6 Astra MarkTechPost Google Hugging Face

Thursday, 3 September 2026

GPT-6 Astra aced the hardest AI benchmark. The asterisk matters more than the score.

The New Stack 1 month ago 34 ● 19 sources

GPT-6 Astra posted a 98.6% result on OpenAI’s ARC-AGI-3 benchmark, far above GPT-5.6 Sol’s 7.8%. The score was evaluated via the Responses API harness with two settings changed for real-world use, and OpenAI notes other models used different setups. The result broadens Astra’s performance claims across other benchmarks and is paired with math progress reports, while OpenAI stresses that the ARC-AGI-3 setup makes direct comparisons less straightforward.

Read the full story

OpenAI Large Language Models AI Benchmarking Model Evaluation

Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps

Hugging Face 1 month ago 33

LFM2.5-350M was fine-tuned with GRPO using TRL for structured-output compliance and then re-evaluated on IFStruct. The run used about 500 samples and 100 training steps, improving the IFStruct pass rate from 22.6% to 29.7%. JSON compliance increased from 18.0% to 31.9% while YAML stayed roughly flat, indicating task-specific fine-tuning shifts the model toward the targeted output formats.

Read the full story

Large Language Models Model Evaluation Reinforcement Learning Fine-tuning

Every AI story that matters,
in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.