AI #186: The World Takes Notice

Jacob Coxon’s resignation seems to have kicked off a preference cascade, and the whole thing has moved with unsettling speed. The mainstream media noticed. Dario Amodei stepped out to say we must pace the frontier, and Anthropic said it would take the first unilateral step by using embedded investigators. OpenAI then said it would do the same, with Google joining in on safety work.

That shift didn’t just stay inside tech. The public took the cue, and estimates of AI killing everyone roughly doubled, from about 15% to about 30%. Politicians responded the way politicians do: calls for regulation, guardrails, and emergency hearings in Congress. But the biggest danger, in this account, is not only the machines. It’s polarization.

Trump has apparently been pushed into treating existential risk as part of a fight over data centers, with David Sacks, Mark Zuckerberg, and Jensen Huang among the people said to have helped blur that line. That has led to the ugly version of the debate: attacks on warning voices, hit jobs aimed at groups like METR, and heat directed at Effective Altruism. The source is blunt about the likely result — a backfire, even if the targets don’t enjoy it.

The same post also points to other recent AI developments. A model solved a Millennium Prize problem. Anthropic reported on attempts to misuse Claude, including large-scale fraudulent distillation attacks that it says were attempted by all the major Chinese AI labs, alongside claims that several of those labs silently passed huge amounts of user data to Anthropic. Anthropic says it has done a remarkable job containing those threats, though it cannot know it has caught everyone.

There is a lot more coming, and the pace is not slowing. The writer expects things to get faster, weirder, scarier, and more stressful, and says the last two weeks already felt unusually intense. For now, the hope is simple: a lull, before the next round of chaos.