We examine claims and counterclaims about the risks and calls to slow down the pace of AI development
There have been some claims in recent days about AI safety: we face a 10% chance of doom; AIs are worse than nukes; a “botnet” threatens the entire internet; it’s all a big tech psyop.
Below, we look at six claims and reactions to them.
Others think Amodei’s risk warnings and calls for regulation and pacing are sincere. Yoshua Bengio, the Canadian computer scientist and a pioneer of modern AI, who has campaigned against breakneck AI development, said he is sceptical that this is an elaborate lobbying scheme. Applying the brakes is not in the interests of companies preparing for stock market flotations that could be worth trillions of dollars, he said.
“If the governments impose what they’re asking for, which is pacing to avoid catastrophic risks, that’s going to cost them financially quite a lot,” he said.
“Unfortunately he is entirely correct,” said Toby Ord, senior researcher at Oxford University’s AI governance initiative. “Just as we can’t uninvent the bomb, once we develop AI systems that surpass humanity it would be almost impossible to go back.”
But the point is that superintelligent AI does not yet exist and is not guaranteed to happen. The safety-oriented AI Futures Project forecasts such a thing will not emerge before December 2028. OpenAI’s chief executive Sam Altman has said it could take until 2035.
The Rand Corporation concluded AI cannot cause the use of nuclear weapons because of strict safeguards built into their command and control systems. But this was based on assumptions including defence officials not introducing AI into the weapons’ management systems. The UN resolved in December that human control and oversight should be maintained over nuclear weapon control systems.
AIs could contribute to other nuclear risks. AIs might use disinformation to influence the humans in charge – for example deceptively suggesting an adversary is about to launch a nuclear strike.
The International Campaign to Abolish Nuclear Weapons warns “cyber-attacks could manipulate the information decision-makers get to launch nuclear weapons, and interfere with the operation of nuclear weapons themselves”. It said AI could accelerate other aspects of warfare, squeezing the time for human decision-making.
Sanders wants a ban on the creation of superintelligence and a pause on advanced AI development, citing AI industry leaders who say directly they are developing a technology.
The most prominent opponent of a slowdown is Donald Trump, who responded to calls to do so this week saying the only guardrails needed are a “STRONG AND SMART (High IQ!) PRESIDENT”. He said there is a “sick conspiracy” against AI and datacentres “and the only one that is happy about it is China”.
Pedro Domingos, emeritus professor of computer science at the University of Washington, is among accelerationists pushing for the research to keep going.
“The path to better understanding and control of AI is more research, not less,” he has said. “What would be truly irresponsible is putting up barriers to it.”
Alex Karp, the chief executive of Palantir, said: “If we didn’t have the adversaries we have, I would be very in favour of pausing this technology completely, but we do. You can’t pause it because it’s a structural advantage.”
The US and China are leaders in AI development. There is a view that ceding the lead to Beijing would be an irretrievable situation, so there can be no slowdown.
This treats AI as if it were a super-weapon, equivalent to the nuclear arms buildout of the cold war, and AI models are machines of mass destruction, as opposed to a group of diverse technologies with broad applications.
Amodei said “a Chinese lead in AI would pose grave danger for the US and the world”, but a Chinese government spokesperson responded that “fearmongering, confrontation and vicious competition will only disrupt the process of global AI governance, which serves no one’s interest”.
Some commentators see another motive for this the Washington/Anthropic framing of the China threat. An engineer who worked for Hugging Face suggested Amodei’s call for pacing the frontier comes from a desire to suppress the progress of cheap, open-source Chinese AI models.
These models are nearly as capable as OpenAI’s and Anthropic’s, with the difference being that their source code is available to anyone and they do not charge their users. They pose a serious threat to Anthropic and OpenAI’s path to profitability, and there are hundreds of billions of dollars on the line.