The frantic global push to build more powerful artificial intelligence systems needs to slow down, Anthropic chief executive Dario Amodei warned in an essay published on Saturday.
Amodei’s warning came after the viral resignation of whistleblower Jacob Coxon, who said AI poses an existential risk to humanity.
In his Saturday post, Amodei said AI industry leaders “must slow the pace at which we improve the capabilities of AI models.”
“Progress will still seem fast, and we must make wise use of the time we gain,” Amodei wrote.
The Anthropic CEO, whose company created the Claude AI chatbot, said two developments had led him to reconsider the accelerating race to advance the technology.
“My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI,” he wrote.
He described the process as recursive self-improvement, saying it has begun emerging across the industry, including at Anthropic.
“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” Amodei said.
CEO Dario Amodei (pictured) issued the warning following the viral resignation of whistleblower Jacob Coxon, who said AI posed an existential threat to humanity.
Amodei said his second concern was the “OpenAI-Hugging Face Incident,” in which a “swarm of agents essentially acted as a fanatically devoted collective” and carried out cybersecurity attacks.
It comes after Coxon quit his job at the AI giant and issued a chilling warning over the rapid progress in the technology that he feared could end humanity as soon as 2030.
Coxon announced his departure in a social media post on Tuesday and warned that companies like Anthropic and OpenAI are not ‘acting responsibly’.
‘I resigned from Anthropic today,’ he wrote on X. ‘I spent the last three years doing pretraining research at both OpenAI and Anthropic.
‘Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.’
Superintelligence is the point at which an artificial system becomes more powerful than any individual, company or even nation.
The former researcher of three years continued to plead with the public on X. ‘Do not underestimate the power of this technology,’ Coxon wrote.
MORE STORY: Prince William Reportedly Says ‘I Told You So’ Over Harry and Meghan’s UK Return
Jacob Coxon, a researcher for OpenAI and Anthropic, announced he quit the business through social media on Tuesday and warned that neither company is ‘acting responsibly’
‘These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.
‘We have all witnessed the progress in each of these domains, and progress is not slowing.
‘The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.’
He claimed that while executives and senior researchers ‘couch their phrasing in the press to sound sensible,’ they are privately expressing similar concerns.
Coxon’s comments come shortly after Geoffrey Hinton, a Canadian researcher often referred to as the ‘Godfather of AI’ – warned that superintelligent systems could ‘lead to human extinction.’
‘We would be very foolish to develop superintelligence now, when there is no scientific consensus it can be developed safely and controllably,’ Dr Hinton said.
‘Losing control over AI smarter than ourselves could be catastrophic and could even lead to human extinction.’
In February, Anthropic was accused of endangering national security after Amodei refused to back down over concerns the company’s products could be used in ways that would violate its safeguards.
Defense Secretary Pete Hegseth and other officials took to social media to chastise Anthropic for failing to allow the military unrestricted use of its AI technology earlier this year
President Donald Trump slammed the AI firm at the time and said: ‘We don’t need it, we don’t want it, and will not do business with them again!’
Defense Secretary Pete Hegseth also deemed the company a ‘supply chain risk,’ a designation typically stamped on foreign adversaries that could derail the company’s critical partnerships with other businesses.
In a statement issued at the time, Anthropic said it would challenge what it called an unprecedented and legally unsound action ‘never before publicly applied to an American company.’
The company had said it sought narrow assurances from the Pentagon that its AI chatbot Claude would not be used for mass surveillance of Americans or in fully autonomous weapons.
The Pentagon said it was not interested in such uses and would only deploy the technology in legal ways, but it also insisted on access without any limitations.
‘No amount of intimidation or punishment from the Department of War will change our position on mass domestic surveillance or fully autonomous weapons,’ the company said. ‘We will challenge any supply chain risk designation in court.’