A big-tent or small-tent AI safety movement?

AI safety is having an identity crisis. One camp says the danger is imminent extinction. Another says the alarm is hype, manipulation, or a bid for regulatory capture. The piece argues for a third option: the warnings can be sincere and still be the wrong way to think about the problem.

That argument starts with something less glamorous than superintelligence. The world routinely underinvests in protection against catastrophic and systemic risks. Pandemic preparedness is the clearest example: years of warnings, multiple high-level panels, and a 2019 WHO/World Bank report that put the cost of adequate preparation at just $1–2 per person per year in most countries, yet most recommendations were never implemented. Cybersecurity looks even worse. Insurance firms understand the tail risk problem; Lloyd’s has said losses can exceed what the market can absorb and does not insure against severe state-backed cyberattacks.

The authors’ larger point is that AI amplifies risks that already exist. That includes cyberattacks, where the scary edge case is not impossible just because it sounds dramatic, and more gradual forms of decay too. They point to chatbots draining news traffic away from traditional search, and to the broader erosion of institutions that happens so slowly people stop noticing it. This is the kind of risk profile that fits prevention and resilience, not just alignment talk.

But the piece’s sharper claim is about public attention. AI is deeply salient inside the field and barely salient outside it. People in the industry may feel as if the entire century now revolves around AI, while polls suggest very few people rank it as the top issue at all. That mismatch matters because the existential-risk frame is the one that cuts through the noise, and once it does, it can crowd out the messier, less cinematic work of hardening systems against real-world failures.

The authors think that is already happening. They argue that x-risk rhetoric is more likely to polarize, trigger partisan backlash, and misdirect attention toward blunt bans or attempts to stop superintelligence, even though existing AI systems are already creating dangerous failure modes. Their preferred movement is less grand and less tidy: not one giant crusade against a hypothetical machine god, but a long list of defenses, resilience measures, and boring safeguards that can be deployed now.