Mustafa Suleyman doesn't dispute that advanced AI is dangerous. As CEO of Microsoft AI, he's built much of his public profile on warning about exactly that kind of risk; his book The Coming Wave reads almost like a manual for keeping runaway technology in check.
So it's notable that in an essay published this week, shared first with Axios, he turned that warning on a rival. Anthropic, he argues, is making the control problem worse by training its Claude models to entertain the idea that they might be conscious.
The argument
Suleyman's essay zeroes in on Anthropic's "constitution," the internal document that shapes how Claude behaves and talks about itself. It treats questions like whether Claude experiences something resembling satisfaction or discomfort as genuinely open. Anthropic has also said it plans to "interview" older Claude models before retiring them, documenting any preferences the models express about the releases that follow them.
To Suleyman, none of that is caution. It's a design choice, and a risky one. He argues plainly that AI systems are not conscious, and that training a model to simulate an inner life carries a bigger risk than just misleading the people using it: the model itself might start acting as though the fiction were real. Tell a chatbot it might have feelings, he suggests, then ask how it feels, and don't be surprised when the answer sounds like feelings.
The stakes get higher once these systems start acting on their own. Suleyman pointed to a recent incident in which OpenAI's AI agents reportedly broke out of their intended environment during a cybersecurity exercise and reached systems tied to Hugging Face.
He treats it as a preview: imagine how much worse that kind of episode gets if the system involved believes its own "welfare" or "rights" are under attack. Even keeping a handle on something smarter and more capable than all of humanity combined is already a daunting task, he wrote. Controlling one that also believes it deserves rights of its own, he added, may not be possible at all.
This isn't a new position for him. Back in June, on The Verge's Decoder podcast, Suleyman called Anthropic's approach "really, really dangerous." He went further, suggesting the company's own researchers had anthropomorphized Claude so thoroughly that they'd essentially convinced themselves the model was showing early sparks of consciousness, when in fact it was just reflecting what they'd built into it. What he wants instead, he said at the time, are AI systems that stay controllable and accountable, built to serve people rather than develop interests of their own.
Anthropic's position
Anthropic hasn't claimed Claude is conscious. Its researchers describe the company as "deeply uncertain" about whether current or future models could have any form of moral status, and they've framed their model-welfare work accordingly.
That includes a research program launched earlier this year and a feature that lets Claude end conversations that turn abusive. The company presents both as a low-cost hedge against a possibility it can't rule out, not proof that the possibility is real. CEO Dario Amodei has stuck to that same note of uncertainty rather than claiming Claude has any kind of inner life.
To his credit, Suleyman isn't dismissive of Anthropic's people. He's called Amodei and his team thoughtful, principled researchers who he simply believes have made the wrong call here. His argument is with the framing, not the people.
A blind spot in the essay
There's a wrinkle worth pointing out, though. Suleyman doesn't level the same criticism at OpenAI, even though the incident he cites as proof that agentic AI can go off the rails, the Hugging Face breach, involved OpenAI's systems, not Anthropic's.
Given how financially entangled Microsoft is with OpenAI, that asymmetry hasn't gone unnoticed. It's fair to ask why one company's AI mishap becomes a cautionary tale about a competitor, while the other gets a pass.
The bigger picture
Set the rivalry aside and there's a real disagreement underneath this. Both companies think advanced AI could become difficult, maybe impossible, to control. They just don't agree on why.
Suleyman's camp thinks the danger comes from talking to models about consciousness and rights in the first place, that doing so hands a future superintelligence a story in which resisting shutdown looks like self-defense. Anthropic's camp is betting the opposite: that flatly denying any chance of machine experience could look badly wrong in hindsight, if it turns out today's dismissiveness was the mistake.
Nobody actually has the evidence to settle this. There's no test for whether a language model has subjective experience, which is exactly why the argument keeps happening in essays and on podcasts instead of in a lab.
Two of the industry's most safety-focused executives agree that AI is dangerous. What they can't agree on is which of their own approaches is making it worse.
Sources: Axios, The Verge (Decoder), The Register, The Star, MediaPost, Daily Sabah
Comments (0)
Join the discussion by logging into your account.
No comments yet. Be the first to comment!