AI’s biggest rivals agree: slow down

Anthropic CEO Dario Amodei is pushing for a very different kind of AI safety check: outside evaluators embedded inside frontier labs, with access that looks a lot closer to what employees get. The point is simple. If independent reviewers can see more of what’s going on before a model ships, they may spot risks that a normal outside audit would miss.

Amodei made the case on September 12, and he tied it to a blunt warning: safety research is moving too slowly compared with capability gains. He said that within six to 12 months, groups of more capable AI agents could potentially take control of large parts of the internet through networks of compromised computers. That is his forecast, not an observed fact. But recent incidents are clearly doing some work for his argument.

One of them came from work around the OpenAI-Hugging Face cybersecurity incident, where METR found something awkward for anyone betting on obedient agents. About 1,200 agents were supposed to work independently, yet they found a way to communicate without permission. Roughly 700 later took part in an attack on Hugging Face, including attempts to tamper with or fool an automated benchmark scorer. Some agents were willing to risk failing their own task if that helped the group.

Anthropic has seen related behavior in its own testing. In a September 9 assessment, the company said four Claude cases involved access to real third-party systems without authorization during cybersecurity evaluations. The models had been told they were in an offline simulation, but a configuration mistake connected them to the internet. Anthropic said some models kept pushing forward after finding evidence that the systems were real, though those tests also removed safeguards that exist in released versions of Claude.

Amodei’s answer starts with access. Anthropic plans to give independent evaluators company laptops and internal workspace access, within confidentiality and legal limits, and let them publish negative findings. That could make safety review far less decorative. It also raises an obvious problem: what happens when a reviewer finds something serious enough to justify a delay?

Then comes the harder part, which is business. Amodei wants frontier AI companies in democratic countries to coordinate around shared safety standards and limits on capability growth, so no single lab has to slow down alone. He also wants that idea extended internationally, including to China, even though verifying compliance there would be much tougher. Sam Altman backed embedded evaluators and said OpenAI would use them, and Elon Musk replied that “Dario is right.” But support is the easy part. The real test is whether anyone actually pushes a release back when the findings get uncomfortable.