TLDRocket
Sign in

AI Benchmarking

291 summarised stories about AI Benchmarking, each linking back to the original source. Browse all topics →

+ Follow this topic

Friday, 28 August 2026

How Decathlon runs demand forecasting at scale with Chronos-2

Amazon Web Services 1 month ago 36

Decathlon selected Chronos-2 as the core model in its demand forecasting stack after benchmarking multiple time series foundation models on its retail data and testing it in production on AWS. Chronos-2 fine-tuned runs with inference under 2 minutes per cutoff for 25,000 products. Decathlon now forecasts weekly on 12-week replenishment and 52-week horizons with less retraining and faster region rollout, reducing WAPE versus its prior models and improving inventory savings and availability.

The Open ASR Leaderboard Adds Its First Global South Language

Hugging Face 1 month ago 23

The Open ASR Leaderboard added two new evaluation sets, Monsoon en-IN and Monsoon hi-IN, to expand ASR testing to Hindi and Indian English with speaker-disjoint public and private splits. Monsoon contains 4,888 speakers across the four splits. Developers can now score models on these datasets (and do private-split evaluation without tuning to the leaderboard), making it easier to measure and reduce performance gaps that are not visible in other leaderboard tests.

Agent Seer: Synthesizing Scenarios from Specification Understanding

Apple Machine Learning Research 1 month ago 50

Agent Seer was introduced to synthesize realistic multi-turn tool-use evaluation scenarios directly from a single MCP tool specification instead of hand-built benchmarks. It was evaluated on seven MCP specifications and achieved complete tool coverage on small and medium specifications. As a result, it produces graded scenarios with synthetic outputs and improves measurements of tool-calling correctness and conversational coherence, highlighting argument value accuracy as the main failure mode and parameter schema complexity as the strongest quality correlate.

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.