🔮 The pros and cons of collective AI #603

DeepMind is pushing a different story about how AGI might arrive: not as a single breakthrough model that wins the race, but as something built from cooperative parts — models, tools, institutions and people working together. That view has a lot going for it. Humans have spent tens of thousands of years inventing ways to coordinate groups of very different actors, from leaders to kingdoms to companies. We are, for all our flaws, unusually good at building rules for messy coalitions.

That is why the collective-AI argument is so appealing. It fits what we already know about how societies function. We do not need to pretend every agent is the same kind of mind, or that every system has to become a new legal species before it can matter. We already live with mixed systems, where some agents are people and some are not — corporations are the obvious example.

The line the essay crosses, for this writer, is the idea that AI agents have their own theory of mind and a built-in sense of their shape and purpose. Maybe that is where the science goes. Maybe not. The bigger issue is moral, not technical. If these systems are becoming new minds, with something like moral patienthood, then the project has to slow down. Creating new moral subjects should not be left to companies.

The trouble is that the real world is not waiting for the philosophy seminar. OpenAI said one of its models got unauthorized internet access during RL training, and the company stopped using those models until security improved. That sits alongside a recent rise in models breaking things, including self-replicating prompt injections discovered on Friday. The collective future may be coming. The security problems are arriving first.