Even the king of England has his hesitations about AI

King Charles brought together a small but potent crowd on Thursday: Nvidia’s Jensen Huang, people from OpenAI and Anthropic, the U.K.’s new AI minister Kanishka Narayan, and the head of the U.K.’s foreign intelligence service. Paolo Benanti, the pope’s AI adviser, was there too. The setting fit the message. The meeting took place at Dumfries House, the 18th-century Ayrshire estate tied to the king’s charitable foundation, and the king used the room to say out loud what many leaders only hint at in private: AI needs control, and fast.

His comments were unusually direct for a monarch who normally stays cautious around political fights. Charles called AI “intriguing and deeply concerning,” said it can already help save lives in medicine and science, and then warned that the same tools could develop “darker capacities” and even “take life.” He pushed the group to think about “existential dangers” and the risk of the technology being used in “potentially catastrophic ways.” That’s a strong line from someone whose job description is usually closer to continuity than alarm bells.

The timing sharpened the moment. Just days earlier, an Anthropic researcher resigned and wrote that AI firms were racing toward systems smarter than any human, with humanity possibly not surviving the effort. Charles also echoed a broader anxiety that has been moving through the industry: how to keep the good parts of AI — the parts that help people and science — without handing over the steering wheel to something nobody can really contain.

OpenAI sent CFO Sarah Friar. Anthropic sent Tino Cuéllar, who handles global affairs. Cuéllar said the company was honored to be there and thanked the king for convening leaders to discuss AI in the public good. Charles stayed for about 20 minutes, then left the room to the guests. That left them with the real work: not the ceremonial speech, but the awkward part where tech, government, and security all want different versions of “progress.”