Agent Evaluation

53 summarised stories about Agent Evaluation, each linking back to the original source. Browse all topics →

Key players

OpenAI Amazon Bedrock Anthropic Hugging Face vLLM Accio team

Monday, 5 October 2026

Evaluating multi-agent systems for explainability and helpfulness with Amazon Bedrock AgentCore

Amazon Web Services 2 weeks ago 41

Amazon Bedrock AgentCore Evaluations was introduced to measure multi-agent systems for helpfulness, task success, instruction following, and explainability beyond just response quality. It supports on-demand and online modes, with online sampling set as a range example of 1–10% of production traces. This enables teams to score agent behavior across accuracy, business-validity checks, and separate explainability signals during development and continuous production monitoring.

Read the full story

AI Infrastructure AI Agents & Workflows AI APIs & Integration Agent Evaluation

Can an Open Model Do Security Research? Cantina’s apex-flash-1 Solves 40 of 60 Held-Out Bug Tasks

MarkTechPost 2 weeks ago 9

Cantina Security and Yeta Labs released apex-flash-1, an open-weights model trained for vulnerability research. It solved 40 of 60 held-out bug tasks (66.7% pass@1) with an estimated $2.38 per run versus $74.68 for Claude Opus 5 High. The release makes a locally deployable security-research “worker” model available under the MIT license, with noted hardware requirements (BF16 needing about 640 GB GPU memory).

Read the full story

AI Security Open-Weight Models Model Releases Agent Evaluation

Every AI story that matters,
in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.