Benchmarking

34 summarised stories about Benchmarking, each linking back to the original source. Browse all topics →

Key players

OpenAI Anthropic Google Together ACL 2026 Findings AI2

Friday, 11 September 2026

Google Research Releases ToolGrad: Answer-First Framework Hits 99.8% Pass Rate for Tool-Use Data Generation

MarkTechPost 1 month ago 12 ● 2 sources

Google Research released ToolGrad, a framework that generates tool-use training data by executing a verified tool chain first and then writing the matching query. ToolGrad raised pass rate from 63.8% to 99.8% on the ToolBench benchmark. This improves training-data efficiency and allows Gemma-3 fine-tunes from 500 samples to reach 83.1 on the Berkeley Function Calling Leaderboard using open code and models.

Read the full story

Google Benchmarking Gemma

Putting Captions to the Test: Evaluating Video Caption Quality through Multiple-Choice Question Answering

Apple Machine Learning Research 1 month ago 17

The article proposes a new way to evaluate video caption quality for Visual Large Language Models that avoids penalizing valid differences in wording or focus. It measures caption quality by running multiple-choice question answering rather than relying on overlap with ground-truth references. As a result, evaluation is reframed around information fidelity and aims to give a more fine-grained quality analysis than existing one-dimensional metrics.

Read the full story

AI Evaluation Video Generation Multimodal Models Benchmarking

Every AI story that matters,
in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.