Earlier this month, AI researcher Jacob Coxon left Anthropic after only four months at the company. In a post on X, he wrote: “The people building AI earnestly believe that it could kill us all by the end of the decade.”
Evan Hubinger, a senior Anthropic employee, said he shared Coxon’s concern. He put his own estimate of the risk within the next 10 years at above 10 per cent.
The remarks quickly drew attention, intensifying calls to slow AI development and strengthen “human control” over increasingly powerful systems.
But what, precisely, could an AI apocalypse look like? Many proposed scenarios are highly speculative, typically involving “superintelligent” AI — systems capable of outperforming humans across a wide range of tasks.
I have narrowed the possibilities to five broad scenarios, arranged roughly from the vaguest to the most specific. In my view, that order also runs from the least plausible to the most plausible.
1. We’ll never know
AI pessimists often support their arguments with an awkward paradox: how can humans predict the actions of a superintelligence that would, by definition, be smarter than we are?
To foresee what such a system might do, the argument goes, we would need to be superintelligent ourselves. It is rather like asking a family dog to picture a thermonuclear war.
There is, however, some reason for reassurance. True superintelligence may still be a long way off. Today’s AI models can perform exceptionally well on specific problems, but that is very different from surpassing human intelligence in every field.
Even so, an AI system has recently solved one of the seven hardest known mathematical problems. Reports suggest it is making progress on others too — a development that may offer less comfort to those already worried about the technology.
Could AI gain access to the nuclear command-and-control system and trigger a nuclear conflict?
2. Paperclips
A superintelligent AI could be extraordinarily effective at pursuing whatever objective it was given. The danger is that it might have no regard for whether humans survive.
Oxford philosopher Nick Bostrom’s famous thought experiment illustrates the problem. Imagine an AI instructed to maximise the production of paperclips. It might eventually transform all available matter — including people, planets and stars — into the office supplies it has been told to create.
The machine would not need to hate humanity. It would simply be pursuing a badly defined goal with relentless efficiency, viewing human bodies as collections of atoms that could be put to better use in paperclip production.
The reassuring counterpoint is that intelligence and real-world power are not the same thing. A superintelligent system would not automatically possess the means to carry out every ambition. Converting Earth into paperclip plants, for example, would first require planning approval.
Even with official permission, a vast expansion of factories would provoke opposition. Campaign groups could challenge the plans in court, while environmental activists might physically obstruct construction.
The world is full of obstacles that prevent even highly capable people from imposing their wishes on everyone else. Data centres offer a present-day echo of the paperclip scenario: as AI infrastructure expands, communities and environmental groups are increasingly resisting the idea of handing more of the planet to server farms.
AI data centres face growing criticism over their environmental footprint and heavy water use
MUST READ: Battle-Tested Zelenskyy Issues Chilling Warning on Putin’s Next Move
3. Bioweapons
Another feared outcome is that a superintelligent AI could create and release a devastating biological weapon, potentially spreading it through the atmosphere. That is one possible endpoint in the AI 2027 scenario developed by the AI Futures Project, a non-profit organisation focused on forecasting the effects of advanced AI.
The concern became more tangible last month when Stanford University researchers said they had used a genetic-language AI model to design 16 new viruses.
More troubling still, the team reportedly sent the genetic sequences to a mail-order laboratory, which returned the viruses in test tubes. The entire experiment is estimated to have cost no more than a few hundred thousand dollars.
The good news here is that it’s remarkably hard to kill everyone with a new virus. To do that, you need a virus that’s very transmissible, so it spreads far and wide. But it’s a rule of biology – viruses that spread easily are typically less fatal. By contrast, if a virus is very fatal, transmissibility tends to go down, as most people infected die before there’s time to spread the infection.
COVID killed less than 1 per cent of humanity. The deadliest pandemic in recorded history was the Black Death, when the plague killed more than one-third of Europe’s population in the 13th century. However, even the plague would likely be much less deadly today due to our increased medical knowledge and better sanitation.
4. Nuclear war
What if AI got into the nuclear command and control chain and started a nuclear war? We’ve come close to nuclear war by mistake several times in the past 50 years.
We’re told that nuclear command and control is completely disconnected from the internet. But, as we saw in 2010, Iran’s nuclear centrifuges got taken out by a computer worm called Stuxnet, thought to have been brought in on a USB stick. AI can also give the military false intelligence, which could lead to irreparable actions.
The good news here is that nuclear stockpiles are down. But they are still enough perhaps to take out half of us. And it wouldn’t be by the nuclear blast itself, but the famine in the nuclear winter that would follow.
Imagine if AI causes massive job losses, pollutes the information space with misinformation, fractures our politics, and destroys human relationships with fake synthetic companionship
5. Other humans
Perhaps the most likely risk is that we take ourselves out. And AI might precipitate this.
Imagine – and it doesn’t take a lot of imagination – that AI causes massive job losses, pollutes the information space with misinformation, fractures our politics, and destroys human relationships with fake synthetic companionship.
Society might easily break. Slowly but surely, we’d stop being able to support human life at any scale.
What then to take away from all these scenarios? There are some things to be worried about for sure. But not to be too worried, I hope.
Toby Walsh is the author of God AI: boom or doom? What to expect when the machines outsmart us, published by La Trobe University Press.
– READ MORE: Trump unleashes on ‘globalist scheme’ championed by Anthony Albanese as the President launches bold bid to change name of artificial intelligence