It sounds like science fiction as some of the biggest names in artificial intelligence warn that the technology their own companies are developing could go seriously wrong without an urgent slowdown.
Over the weekend, Anthropic boss Dario Amodei published an online essay calling for a coordinated reduction in the pace of AI development to give companies and governments more time to establish better guardrails.
While those concerns have existed for years, Amodei said events in recent months had convinced him that action was needed.
Some experts have described that incident as a wake-up call that provided a concrete demonstration of the safety risks they had long warned about.
Among Amodei's concerns was so-called "recursive self-improvement", where AI becomes capable of improving AI, potentially creating a cycle of powerful systems that outrun human ability to understand or control them.
The leaders of two rival AI companies, OpenAI's Sam Altman and xAI's Elon Musk, both voiced their agreement.
Alarm about the potentially existential risks posed by AI grew after an Anthropic researcher resigned last week, stating that "people building AI earnestly believe that it could kill us all by the end of the decade".
United States President Donald Trump rejected calls to check the development of AI and suggestions that the technology is moving too quickly, attributing those warnings to "negative forces".
Geoff Webb, an Australian laureate fellow in the Monash University Department of Data Science and Artificial Intelligence, told SBS News he doesn't think it's "remotely feasible" that AI companies will pace their development.