On 9 September 2026, a 27-year-old researcher called Jacob Coxon quit his job in the most public way possible: in a thread on X. Coxon had spent three years training AI models at OpenAI and then Anthropic, two of the companies racing to build the most powerful systems on the planet. He wasn't leaving over pay, burnout or a difficult manager. He was leaving, he wrote, because the people building AI genuinely believe it could wipe out humanity before the end of the decade. Within a day, his post had been seen tens of millions of times.

Dramatic resignation letters are usually easy to dismiss. This one wasn't, because of who replied. Evan Hubinger, who leads alignment science at Anthropic, the team whose whole job is making sure AI does what humans actually intend, didn't push back. “We really do earnestly believe AI could kill all humans,” he wrote, adding that his own estimate was more than ten per cent within the next ten years. He also admitted that nobody yet has a plan for keeping a superintelligent system under control.

Put that number into everyday terms. If an airline told you that one flight in ten would crash, you wouldn't get on the plane. If a new medicine killed one patient in ten, it would never reach a chemist's shelf. Yet in AI circles, ten per cent isn't a fringe figure. Geoffrey Hinton, the Nobel Prize-winning scientist often called the godfather of AI, told BBC Radio 4 in December 2024 that he saw a 10 to 20 per cent chance of AI leading to human extinction within thirty years. A 2023 survey of 2,778 AI researchers found the typical respondent put the risk at around five per cent, and somewhere between a third and a half of them put it at ten per cent or higher.

Not everyone is convinced. Critics point out that these figures are educated guesses dressed up as maths, and that talk of the apocalypse distracts from the harm AI is already doing, from lost jobs to misinformation. Some go further and suggest doom talk quietly flatters the industry: a technology powerful enough to end civilisation must surely be worth a fortune. Yann LeCun, one of the field’s best-known sceptics, has long argued that the fears are wildly overblown.

Politicians, though, are no longer waiting to see who is right. In the same week, US Senator Bernie Sanders pointed to the row as proof that the people building the technology admit it could threaten humanity’s future, and lawmakers on both sides of the Atlantic have put forward bills to ban the development of superintelligence altogether. Whether a ban could ever work is another matter. Once one company or one country builds something, the genie is out of the bottle, and we are in genuinely uncharted territory.

Which leaves the question underneath all the statistics. We accept small risks every day: we drive, we fly, we cross busy roads. But those are risks we choose for ourselves. Is it acceptable for a handful of companies to take a one-in-ten gamble on behalf of eight billion people who were never asked? And if the people closest to this technology are really this worried, why are they still building it?