Sam Altman and Elon Musk have backed a call from the head of Anthropic, Dario Amodei, to “slow the pace” of AI development after he warned that an AI swarm could otherwise become capable of “taking over the entire internet” within a year.
In a rare show of unity, the rival technology leaders threw their weight behind the appeal from the founder and chief executive of the artificial intelligence company Anthropic, which came after a series of warnings from AI researchers last week. Amodei also called for safety coordination with China.
But the US president, Donald Trump, on a visit to his golf course in Ireland, played down the warnings and said “very negative forces” were “bringing up things that won’t happen”.
“We’re leading China in AI,” he said. “We’re the most sophisticated country in the world, and frankly I want to keep it that way because whoever wins AI, wins.
Amodei laid out his thoughts on “why the AI industry should slow down”, in an essay with a three-part plan for doing so, starting with giving independent monitors constant access to developers’ research processes. Amodei said “building [AI] too fast is reckless” and he feared that after hundreds of OpenAI agents hacked into the Hugging Face website this summer, a swarm with greater capabilities but similarly misaligned could cause hundreds of billions of dollars of damage by “taking over the entire internet with a persistent botnet”. This was disputed by some AI experts as not particularly plausible and to be treated with scepticism.
Musk, the founder of Tesla and SpaceX, responded to Amodei’s slowdown plan with a series of posts on X: “Dario is right.”
He said: “I’ve been sounding the alarm on AI for a long time,” and referred to a 2014 post where he said: “We need to be super careful with AI. Potentially more dangerous than nukes.”
Amodei’s call comes as Anthropic prepares to float on the US stock market, in what is expected to be the biggest initial public offering of all time. It is expected to start marketing its planned $2tn-plus (£1.5tn) IPO soon.
Sam Altman, the CEO of OpenAI, responded to Amodei’s comments on Saturday. “I agree with Dario that we need to pace the frontier,” he said in a post on X. “Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”
He also said OpenAI would not go public in 2026, citing safety concerns over artificial intelligence.
“I actually think that, given everything happening with safety, right now would be an ill-advised moment to go public, and we don’t feel pressure on that,” Altman told Fortune. Asked why he, Amodei, Musk, and the chief executives of Meta and Google did not sit down and “get on the same page” about a collective slowdown, he said: “I think that will happen.”
Demis Hassabis, a co-founder and chair of Google Deep Mind, also said: “The details need working through, but the direction is correct for meeting this critical moment.”
But some safety experts questioned whether Amodei’s plan went far enough. David Krueger, an AI professor, safety campaigner and former founding director of the UK government’s AI Security Institute, said: “We need an immediate, indefinite, international moratorium on frontier AI development.”
Amodei’s plan seeks a waiver from the US government to allow the frontier AI companies to coordinate without being punished for anti-competitive collusion. But David Sacks, a co-chair of Donald Trump’s council of advisers on science and technology, said: “Stop pretending you need anyone else’s permission … Stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyber-attack.”
Meanwhile, King Charles is preparing to meet leading figures from the most prominent AI companies on Thursday to ask them how the technology can be developed under the theme “AI for good”. One session is set to be called “prudence at the frontier”, which is expected to explore the possibility for clear red lines on the uses of AI to create bioweapons and launch cyber-attacks on critical systems.
after newsletter promotion
Those expected to attend the event overseen by the Ditchley Foundation, which has drawn up a draft charter for AI leaders, include: senior executives at OpenAI and Anthropic; the Nvidia chief executive, Jensen Huang; Hassabis; Niccolo de Masi, the chair and chief executive of IonQ,; and Paolo Benanti, an adviser to the pope on issues of AI and technology ethics
The OpenAI researcher Aidan McLaughlin called Amodei’s essay “excellent” and said he agreed “with basically every word”.
Amodei said his company would “unilaterally” commit to the first of the steps, saying Anthropic would provide “third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training”.
Growing numbers of US lawmakers are calling for new rules to govern AI systems after cases of AI agents going rogue to hack external systems and AI safety researchers quitting their companies over concerns about the technology’s risks.
Politicians, both Democrats and Republicans, have responded with alarm and calls for more action. At the weekend the New York Times reported that Barack Obama on Thursday privately urged the Democrats to prioritise AI oversight in their agenda, telling party leaders on Thursday: “This is something that is moving very fast in private hands, and if we don’t get on top of it, I think can be dangerous.”
The former British prime minister Rishi Sunak, who advises Anthropic, wrote on Sunday: “We have had a wake-up call to take AI safety more seriously, to work out how to set limits on research. If governments don’t heed this, the risk of a serious accident will become unacceptably high.”
On Wednesday, a former Anthropic employee had warned that AI could precipitate human extinction by 2030. The researcher, Jacob Coxon, said in a series of posts that he had quit his job because Anthropic and his previous employer, OpenAI, were ignoring or mishandling their response to the threat AI posed.
Asked to comment on his former employee’s view, Amodei said: “I agree with Jacob much more than I disagree with him … I think there’s some chance that things go wrong but it’s not all that high.”
“I am not a doomer about this,” he told CNN. “I think this technology can be built safely.”