Anthropic pitches AI safety bill that may block a model if 50+ people could be killed
TechCrunch Lucas Ropek ● Covered by 8 sources
Anthropic wants a bill that could block an AI model if it may kill 50 people or cause $1 billion in damage. It’d put the Commerce Department, not AI labs, in charge of stopping a release.
Based on reporting by TechCrunch, Lucas Ropek — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Anthropic is pushing its own answer to the AI safety fight: a draft bill that would give the U.S. government a formal way to stop a model from being released. Politico reported that the company has been shopping the proposal around in hopes Congress might take it up.
The bar it sets is unusually specific. A release could be blocked if there is a “foreseeable, imminent risk” that more than 50 people could die, or that more than $1 billion in property damage could follow. The draft says that risk would have to come from a biological weapon, a model escaping its developer’s controls, or a cyberattack.
Anthropic’s idea borrows from California’s recently passed Senate Bill 53, which uses similar thresholds for what counts as “catastrophic risk.” But the company’s bill also tries to define how the danger gets assessed. It calls for “independent evaluators” registered with the U.S. Commerce Department, and those evaluators could be paid by the AI labs as long as payment isn’t tied to the result of the evaluation.
There’s a catch, of course. The draft doesn’t explain how to make sure money can’t quietly sway the judgment. And when evaluators think a model poses an imminent catastrophic risk, they would have 72 hours to alert the Commerce Department, which would then have to go to a D.C. judge to seek a halt. Federal agencies would not be allowed to block the model on their own.
The timing says a lot. The industry’s loudest near-term worry has been agentic hacking, not a cinematic mass-casualty scenario. Yet labs keep floating systems that let them police themselves with “neutral” outsiders embedded inside the business. Anthropic’s version just gives that instinct a more formal jacket and a federal seal.
My take — AI-written commentary, not fact-checked reporting
This is classic AI industry behavior: warn about apocalypse, then propose a process where the industry still gets to pick the referee. If a company wants public trust, “independent evaluators” who can be paid by the labs already sounds like a group project with a sponsor. The real tell is how allergic these firms remain to rules they didn’t write themselves.
Read more about this at: TechCrunch
