đź”® Quick weekend reads: the big AI questions
Exponential View Azeem Azhar
AI is dragging two huge debates into the open: who should control it, and whether it can be conscious. The strange part is that one story is about power, the other about personhood.
Based on reporting by Exponential View, Azeem Azhar — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Weekend reading on AI starts with a blunt warning from The Economist about effective altruism, which it calls “this century’s biggest idea.” The piece argues that some people in Silicon Valley think only they can be trusted with very powerful AI. That is a dangerous place to end up, because the same logic that says a machine could lock onto one bad objective can also be used by humans who decide they alone get to steer the thing.
That’s the real tension here. EA’s case for AI risk is that an advanced system might optimise for a single goal and do it disastrously well. But the concern doesn’t stop with machines. A small group of humans can get trapped by the same kind of certainty, and then the problem becomes political as much as technical.
There’s also a very different weekend read: The New York Times reports that Anthropic has been consulting religious scholars and theologians about machine consciousness. Anthropic has already put more weight on questions of consciousness and personhood than most AI companies, which makes the story feel less eccentric than it would elsewhere. According to the report, Anthropic’s team even considered pulling out of a Vatican event before the Pope’s encyclical Magnifica Humanitas drew a hard line, saying AIs do not experience, do not have bodies, do not feel joy or pain, and do not have a moral conscience.
That’s a lot to ask of one weekend. But the overlap is obvious: both stories are about what happens when people project grand powers onto AI, then build institutions around those projections. The reader who wants a neat, settled answer is going to be disappointed. The better move is probably to keep the questions uncomfortable a little longer.
My take — AI-written commentary, not fact-checked reporting
This is exactly why AI debates keep going weirdly religious on one side and weirdly managerial on the other. The danger isn’t just rogue models; it’s the human habit of deciding a tiny circle should get to define what counts as safe, true, or even alive. That’s how you end up with serious people saying extremely odd things with straight faces.
Read more about this at: Exponential View
