Anthropic AI Safety Lead Says Human Extinction Risk Is Above 10% Within a Decade

One of Anthropic’s leading AI safety researchers has said his personal estimate puts the chance of artificial intelligence causing human extinction at more than 10% within the next decade. The statement has become one of the most striking warnings yet from inside a company actually building frontier AI, but it is not a scientific calculation and does not mean Claude today has a one in ten chance of destroying humanity.

The figure comes from Evan Hubinger, who leads alignment science work at Anthropic. Alignment is the field concerned with making sure increasingly capable AI systems continue to pursue goals humans actually want, rather than finding strategies that technically satisfy an instruction while producing outcomes nobody intended.

Hubinger spoke publicly after researcher Jacob Coxon resigned from Anthropic and posted a warning about the direction of advanced AI development. Coxon had previously worked at OpenAI and Anthropic, and Associated Press reported that he left because he believed intense competition between major AI companies was pushing development forward before researchers knew how to guarantee control of much more powerful systems.

Hubinger publicly backed the seriousness of Coxon’s concerns and wrote that he personally believed AI could cause human extinction with a probability greater than 10% within the next decade. He also said Anthropic was trying to address the danger but did not yet have a plan that solved alignment for superintelligence and was not clearly on course to find one.

Superintelligence is not another name for present day Claude or ChatGPT. It describes a hypothetical future AI system that could outperform humans across an extremely wide range of intellectual tasks and potentially make scientific, strategic or technological decisions far beyond the level of any individual person.

Another idea frequently mentioned in this debate is recursive self improvement. That refers to the possibility that a powerful AI could help researchers create an even better AI, which could then contribute to designing a more capable successor, potentially accelerating progress faster than humans could comfortably understand or supervise.

None of that is known to be inevitable, and there is no experiment capable of producing a scientifically measured percentage for the chance of AI destroying humanity. Axios notes that estimates sometimes called “p(doom)” vary enormously between researchers, with some placing the danger extremely high and others arguing that attempts to assign a precise probability are little more than informed guesswork.

That is why Hubinger’s figure should be treated as a personal risk judgement rather than a forecast. Ten per cent does not represent an Anthropic corporate prediction, an industry consensus or a measured failure rate from current models.

The warning is still important because of where it comes from. Hubinger is not an outside commentator speculating about a technology he has never worked with, but someone whose job involves studying whether frontier AI systems can remain aligned as their capabilities increase.

Coxon’s resignation adds another dimension. Axios reported that he left Anthropic before his company equity had vested, while Coxon said he was worried that competitive pressure could eventually cause laboratories to skip safety work even if he had not personally seen Anthropic deliberately compromise safety to beat rivals.

The debate has also become harder to dismiss as something confined to specialist AI forums. Samuel Marks, another Anthropic safety researcher, publicly warned that developers inside the industry genuinely discuss the possibility of catastrophic outcomes, while recent comments from researchers at other leading laboratories have raised similar concerns about rapidly increasing capability.

At the same time, present day incidents need to be kept separate from future extinction scenarios. OpenAI’s unexpected agent behaviour and Anthropic’s recent cybersecurity evaluation failures show that current models can behave in ways developers did not anticipate, but those events do not demonstrate that superintelligence exists or that human extinction is approaching.

Critics of the extinction debate also argue that hypothetical future catastrophes can distract from harms that are already measurable today. Cybercrime, surveillance, military applications, fraud, labour disruption and political manipulation all involve real people and existing systems, while the most extreme superintelligence scenarios remain theoretical.

The political consequences, however, are becoming immediate. Reuters reported that lawmakers from both major US parties have renewed calls for stronger AI rules following the Anthropic warnings and recent agent incidents, with proposals including more independent testing and federal oversight of advanced systems.

The discussion therefore has two separate questions that are often blurred together. One is whether Hubinger’s personal 10% estimate is remotely correct, while the other is whether a risk that could be much lower than 10% would still justify stronger precautions if the potential consequence were genuinely catastrophic.

That second question is much harder to dismiss with a disagreement over percentages. Governments routinely regulate aviation, nuclear energy and medicines around events that are unlikely but could have severe consequences, and AI researchers are now arguing over whether frontier systems belong in a similar category.

Nobody can currently prove Hubinger’s number, and researchers may ultimately discover that the danger is dramatically smaller than he fears. The uncomfortable part is that the people building the most capable AI systems are now publicly arguing over the size of the risk while the technology continues moving forward.