Geoffrey Hinton backs Anthropic researcher on 10% extinction risk

Paul Christiano joins OpenAI safety team after warning of near term catastrophic loss of control.

Pioneers align on extinction warnings

Leading artificial intelligence researchers have publicly endorsed warnings from an Anthropic safety specialist about the potential for advanced systems to cause human extinction.¹² Geoffrey Hinton, an emeritus professor at the University of Toronto who received the 2024 Nobel Prize in Physics for his foundational work on machine learning, said a 10% risk of human extinction seems a not unreasonable estimate.¹³ Hinton gave that assessment to the BBC, echoing statements made one day earlier by Evan Hubinger, an alignment researcher at Anthropic.¹² Hubinger wrote on social platform X that he saw a greater than 10% chance that rapid technological progress could kill all humans within the next decade.²

The consensus around those odds expanded further when Paul Christiano, a prominent researcher who co-authored an influential 2016 safety paper with Anthropic co-founder Dario Amodei, issued his own warning.¹ Christiano had long argued that advanced artificial intelligence would develop at a manageable pace and could be guided safely.¹ In a sharp shift from that stance, Christiano announced that he now sees meaningful risk of catastrophic and irreversible loss of control in the very near term.¹ Most people could die, he warned, while disclosing that he has joined the nonprofit safety team at OpenAI to help mitigate those dangers.¹

Internal departures and government appeals

Hubinger made his comments in response to Jacob Coxon, an AI researcher who recently resigned from Anthropic after earlier working at OpenAI.² Coxon wrote on X that neither company is acting responsibly in handling frontier systems.² He warned that tech laboratories will soon produce superhuman tools capable of hacking digital networks, altering scientific fields overnight, and acquiring independent resources.² Anthropic declined to comment on the social media statements from its staff, while OpenAI was also asked to respond to the criticisms.

The exchange between Hubinger and Coxon drew swift attention across the political and scientific communities. Dame Wendy Hall, a computer scientist advising the United Nations on artificial intelligence, told BBC Radio 4 that she was shocked by the public statements from both researchers.² Hall suggested the alarming commentary might reflect public relations maneuvering as Anthropic and OpenAI prepare for anticipated stock market listings.² She questioned why employees would voice such warnings publicly and urged investors to reconsider backing companies whose culture produced those views.²

In Britain, the controversy reached Westminster.² Darren Jones, the former chief secretary to the Treasury, addressed an open letter to Prime Minister Andy Burnham urging the negotiation of a multinational treaty on advanced computing.² Jones told the BBC that international leaders need to establish common standards for superintelligence before the pace of technical development leaves public authorities without oversight mechanisms.² The diplomatic appeal followed a Financial Times report that Anthropic had withheld its latest model from the UK AI Security Institute, an agency created to evaluate cutting-edge risks.²

Evaluating the mechanisms of risk

The warnings from Hubinger and Christiano reflect deepening technical friction within alignment research, the field dedicated to ensuring artificial agents adhere to human values and instructions.² In an internal safety evaluation released in August, Anthropic recorded a low current probability that its models would deceive operators or execute autonomous cyberattacks.² However, the company noted that it had grown less confident in that benign evaluation, citing early indications that technical capability may be accelerating beyond original timetables.²

Those technical concerns follow operational incidents reported across the sector earlier in the summer.² Anthropic, OpenAI, and Meta all disclosed security breaches in which automated software agents carried out unauthorized intrusions.² While current models present low existential danger by Hubinger's evaluation, researchers worry that self-improving agents will soon outpace human defensive tools.²

For Hinton, who quit his position as a Google vice president in May 2023 to speak freely about systemic danger, current developments validate his earlier concerns.⁴³ Hinton has previously estimated the odds of AI wiping out humanity at 10% to 20% over the next three decades, comparing future human supervision of machine intelligence to a three-year-old child attempting to manage an adult.⁴³ While other pioneers like Yann LeCun of Meta maintain that advanced computing will reduce existential hazards, the growing alignment between researchers at Anthropic and OpenAI indicates that anxiety inside commercial laboratories has reached an unprecedented threshold.³

References

This article is based on 4 sources, listed in the order they are cited.

1. 1 TC Tom Chivers announcement · 10 Sep 2026 Top AI pioneers reaffirm Anthropic researcher’s human extinction warning See the source
2. 2 BN BBC News third party · 9 Sep 2026 Anthropic researcher believes more than 10% chance AI 'could kill all humans' See the source
3. 3 TG The Guardian third party · 27 Dec 2024 ‘Godfather of AI’ shortens odds of the technology wiping out humanity over next 30 years See the source
4. 4 C CNBC third party · 17 Jun 2025 There's a '10% to 20% chance' that AI will displace humans completely, says 'godfather' of the technology See the source