Anthropic Adds Plugin Evals to Claude Code: 6 Grader Types, a No-Plugin Baseline, and a CI Gate for Skills
MarkTechPost 1 month ago 36
Anthropic added a plugin evals workflow to Claude Code that runs a plugin against prompts, grades the output, and compares it to a no-plugin baseline. It runs twice per case by default, so the key metric is Δ and a $0.41 example took 74 seconds across 6 runs. Results now include per-grader verdicts and can be enforced in CI with a threshold and max-cost limit, revealing cases where skills don’t trigger on natural phrasing.