TLDRocket
Sign in

AI Benchmarking

289 summarised stories about AI Benchmarking, each linking back to the original source. Browse all topics →

+ Follow this topic

Wednesday, 23 September 2026

What it takes to bring up a multi-rack NVIDIA Vera Rubin NVL72 cluster

coreweave.com 3 weeks ago 32 ● 6 sources

CoreWeave describes how multi-rack NVIDIA Vera Rubin NVL72 clusters are brought up by tightly coordinating compute, networking, power, and liquid cooling with automated validation to avoid distributed-system bottlenecks. It says each Rubin GPU is served by two ConnectX-9 SuperNICs delivering up to 1.6 Tb/s backend connectivity per GPU. As racks scale out, the process shifts from per-node checks to rack-, fabric-, and synchronized workload testing so stragglers and underperforming links are caught before production.

How to Evaluate AI Agents: From Tool Calls to Task Completion

NVIDIA Technical Blog 3 weeks ago 2

AI agent evaluation has shifted from grading single model outputs to grading complete task execution that includes sequential tool calls, environment state tracking, and recovery after failures. The piece highlights SWE-bench Verified traces where the reported outcome score is based on end-to-end checks while the process is scored step-by-step, including an example E2E result of 1 with step-level scoring of 3/4. Benchmarks and release gates now prioritize tool-use and environment-based verification (e.g., state changes and tests) alongside process traces, rather than treating call text accuracy as sufficient.

Introducing MentalHealthBench

OpenAI 3 weeks ago 45

MentalHealthBench was introduced as an expert-informed benchmark for evaluating helpful and safe AI responses in realistic mental health conversations. The article does not provide any numeric benchmark, date, or other concrete figure. This changes how developers can measure and compare AI response quality for mental health dialogue, though the piece itself contains only a brief description.

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.