As AI giants warn of existential risk, they can also build a competitive moat via safety. 'It’s genius and they’re all going to make a lot of money'

A strange thing is happening in AI: the biggest players are talking more like alarmed regulators than product companies. Anthropic and OpenAI have both argued that frontier models are so powerful they should be tested independently before release, and they’ve done it in essays, speeches and social posts that frame the technology as a public danger as much as a commercial prize.

That message lands in a very specific moment. The companies are heading toward much-anticipated public listings, the stock market is jittery, and Washington is still loose enough on AI that the labs get to define a lot of the safety conversation themselves. President Donald Trump has waved away warnings about AI risk as a “HOAX” and has shown little appetite for fresh rules, while his administration already relies on the U.S. Center for AI Standards and Innovation, a federal agency created in 2023 to test models voluntarily.

But the safety pitch is not just about public service. Former OpenAI geopolitics lead Sarah Shoker says the focus on existential doom pushes aside messier, present-day problems such as data-center pollution, hacking, surveillance and military use. Those are real issues now. Still, the biggest companies benefit when the debate stays on grand, abstract threats rather than on what their systems are already doing in the wild.

The labs are also trying to set the rules for how they themselves will be judged. There are no universal standards for testing AI the way there are in restaurants, finance or aviation, and the companies are not asking for more power at the federal agency that already exists. Instead, they want their own auditing frameworks and their own chosen evaluators, including outside groups like Berkeley-based METR. That gives them flexibility, but it also gives them control.

Investors can see the shape of the deal. Pitchbook analyst Harrison Rolfes said the big labs are making themselves look like the safest bets in the sector, which can help them raise capital, please chip suppliers and shut smaller rivals out. Or, as he put it, they may be building a wall and calling it safety. The rough part is that the warning may be genuine and the business benefit may be just as genuine.