HarnessDev tested whether LLMs can write and iteratively improve the agent “harness” (the runnable code with tools and execution loop) rather than just produce answers. Only 34 out of 64 harness changes moved held-out scores in the same direction as feedback, with execution gains described as small and noisy. The work concludes harness quality is executor-specific and that much generated state/memory code often never runs, so generalization across executors and revisions is limited.
Dynatrace acquired Arize AI to bring AI observability, evaluation, and agent monitoring into Dynatrace’s application observability platform. Arize said its open-source Phoenix platform is used by more than 4,000 enterprises. The combined offering shifts observability from spotting issues to providing shared context so teams and AI agents can diagnose and take action on problems.
Anthropic added a plugin evals workflow to Claude Code that runs a plugin against prompts, grades the output, and compares it to a no-plugin baseline. It runs twice per case by default, so the key metric is Δ and a $0.41 example took 74 seconds across 6 runs. Results now include per-grader verdicts and can be enforced in CI with a threshold and max-cost limit, revealing cases where skills don’t trigger on natural phrasing.
Amazon Bedrock AgentCore Evaluations and AWS DevOps Agent were used to monitor a production multi-agent airline reservation system, covering both agent-quality failures and infrastructure incidents that traditional CloudWatch checks can miss. The system uses a sampled percentage of production requests for background scoring via LLM-as-a-Judge. Together, they add continuous quality metrics plus autonomous root-cause investigations and remediation recommendations, creating a continuous feedback loop for improving and deploying the agents.
A benchmarking harness measured accuracy and real “cost per outcome” for five OpenAI models, comparing OpenAI API baselines (gpt-5.4-mini, gpt-5.4-nano) with three OpenAI models on Amazon Bedrock (gpt-5.6-luna, gpt-5.6-terra, gpt-5.6-sol) across single-call tests, multi-turn agent trajectories, and rubric-graded professional deliverables. On July 30, 2026 Bedrock repricing, gpt-5.6-luna recorded $0.0021 per correct AIME answer versus $0.0139 for gpt-5.4-mini, and $0.010 per passing GDPval deliverable versus $0.030 for gpt-5.4-mini. Model choice shifts from comparing token prices to comparing measured cost per successful outcome under the right workload shape, especially accounting for turn efficiency in agents and per-deliverable quality gates.
Cognition helped Devin use GPT-6 Astra to test its own software work and demonstrate it functions. The update is GPT-6 Astra. As a result, Devin’s output shifts toward more self-verification, so engineers can review less code and ship more.
Datasette released two security patch versions, 1.0a39 and 0.65.4, for public web deployments. The update specifically includes a stable fix for the 0.65.x family. The releases incorporate fixes for subtle bugs found during an AI-assisted security audit and change future development by adding frontier-model security audits.
The article proposes a new way to evaluate video caption quality for Visual Large Language Models that avoids penalizing valid differences in wording or focus. It measures caption quality by running multiple-choice question answering rather than relying on overlap with ground-truth references. As a result, evaluation is reframed around information fidelity and aims to give a more fine-grained quality analysis than existing one-dimensional metrics.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
