Amazon released SOP-Bench, an openly available benchmark for scoring AI agents on real enterprise SOPs executed with working tools and graded against ground-truth outcomes. The benchmark includes 2,000+ tasks across 12 business areas and was presented at the 2026 KDD conference. Results on 11 frontier models showed performance can drop after model upgrades, that adding extra tools can nearly halve success, and that agents need step-level oversight rather than relying on a single overall benchmark score.
Mobility-Embedded POIs (ME-POIs) introduced a mobility-informed framework that turns language-model place representations into embeddings capturing places’ temporal activity rhythms. It reported up to an 81.9% relative gain in predicting visit intent, a 75.1% improvement in price level classification, and a 24.7% increase in busyness estimation accuracy on unseen places. As a result, AI can better infer real-world attributes like opening hours, price levels, and busyness from aggregate mobility plus text, including for sparse places, without enabling individual personalization.
Glenn Matlin and Chandreyi Chakraborty traced where an Olmo 3 language model’s social reasoning comes from by estimating how different training-document categories influenced benchmark answers. They sampled about 5.68 million documents from Dolma 3 (which contains about 1.26 billion documents) and found SocialIQA relied much more on narrative, interpersonal writing categories than other benchmarks. When they deleted the most influential literature data, the model’s SocialIQA score fell more than after removing random documents, suggesting training-data changes can measurably affect social reasoning ability.
DeepSWE head-to-head testing compared GLM-5.3 and GPT-5.6 Sol, then built a two-model cascade that routes from GLM-5.3 to Sol when tests fail. The cascade solved 85.9% of tasks at $6.61 per task, beating Sol alone at 72.7% and $8.37 per task. Deployments should start with GLM-5.3 as the lower-cost first pass and escalate to Sol via verification to improve solve rate for less money.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
