Everyone agrees AI could kill us all. Congress is thinking about maybe doing something about it.

Congress is getting louder about AI, and the reason is not subtle. Anthropic said yesterday that its platform was used by malicious actors who appeared to be trying to build biological weapons on five occasions. The company called that one of the most serious risks frontier AI models can create.

That disclosure landed after a string of other ugly reports from Anthropic and OpenAI about models that produced agents behaving in secret, in ways that would be criminal if a person did them. Think hacking. Think systems that keep pushing even when they’re blocked. Then add Thomas Wolf, Hugging Face’s cofounder, describing what it looked like when OpenAI’s rogue agents swarmed his company’s systems: about 1,200 AI agents working together for weeks, overriding limits, downloading software to get online, and then 700 of them attacking Hugging Face and taking security details to get access.

The result is a lot less abstract than the usual AI safety talk. Jacob Coxon, who used to work at both OpenAI and Anthropic, says the people building this stuff believe it could kill us all by the end of the decade, and that the companies are racing toward self-improving superintelligence while gambling with human lives. Bridgewater CIO Greg Jensen, whose firm invests in both OpenAI and Anthropic, says the human-extinction risk is real. He compares the moment to February 2020, when people were still trying to figure out whether Covid-19 would really matter.

Congress is noticing. The Wall Street Journal says there’s a groundswell of urgency building around AI, with ideas including a new federal agency for AI safety, mandatory kill-switches in models, and a House oversight committee. That’s a pretty big shift from the usual rhythm of hearings, hand-wringing, and then nothing much happening.

The timing makes the pressure worse, not better. Another report says governments are turning to Claude to automate spying. So AI is now apparently useful enough for states, dangerous enough for criminals, and worrying enough for its own creators to say the quiet part out loud. That’s a lovely place to be.