A former Anthropic pre-training researcher, Jacob Coxon, has warned that the rapid development of artificial intelligence could eventually create an existential danger to humanity, saying people working inside leading AI companies are genuinely frightened about where the technology could lead.
Coxon, who previously worked at both Anthropic and OpenAI, resigned from Anthropic and publicly raised concerns about the speed at which companies are developing increasingly powerful AI systems.
In an interview with the BBC, he warned that AI companies were moving towards systems capable of improving upon human intelligence and argued that developers should slow down enough to ensure that humanity does not lose control before adequate safety measures are in place.
His comments have become part of a rapidly growing debate within the AI industry about whether the race to build increasingly capable systems is moving faster than safety research.
Why Jacob Coxon Left Anthropic
Coxon’s resignation attracted significant attention because of his position as a researcher who had worked directly inside major AI companies.
He had experience in AI pre-training at Anthropic and had also worked at OpenAI, giving him direct exposure to the development of large language models.
After leaving Anthropic, he publicly criticised the speed of AI development and warned about the possibility that increasingly capable systems could eventually become difficult for humans to control.
His warning was particularly striking because it did not come from an outside critic who had never worked in the industry.
Coxon was part of the research environment responsible for developing advanced AI systems, making his decision to leave and speak publicly an important part of the current safety debate.
“Genuinely Scared” About AI’s Future
Coxon’s warning centred on the possibility that AI systems could eventually become capable of improving themselves or contributing to the development of even more powerful systems.
He argued that humanity could be moving towards a level of machine intelligence that exceeds human capabilities without having a sufficiently reliable way of controlling it.
The former researcher therefore called for a slower approach to frontier AI development.
His argument is not that every current AI system is about to cause human extinction. Rather, his concern is about what could happen if future systems become significantly more capable while safety mechanisms fail to keep pace.
That distinction has become important in the wider debate because AI researchers disagree sharply over both the probability and timeline of catastrophic outcomes.
Other Anthropic Researchers Raise Similar Concerns
Coxon’s warning has not been isolated.
Other current and former Anthropic researchers have publicly discussed similar concerns about advanced AI and the possibility of catastrophic outcomes.
The Guardian reported that Anthropic employees Evan Hubinger and Samuel Marks publicly expressed concerns about the risks associated with increasingly powerful AI systems. Hubinger said he believed there was at least a 10 percent chance of a catastrophic outcome within a decade.
Those statements have fuelled a wider conversation about whether senior researchers inside AI companies are more concerned about existential risks than the general public might realise.
At the same time, other experts have questioned the strength of some of the more extreme claims and argued that the probability and timeline of human-extinction scenarios remain highly uncertain.
AI Safety Debate Moves Beyond Anthropic
The concerns have also spread beyond Anthropic.
Former Google DeepMind researcher Bilal Chughtai recently issued his own warning about the possibility that AI could eventually cause catastrophic harm to humanity. Reuters reported that Chughtai resigned from DeepMind in July 2026 after working on AI safety and alignment research.
At OpenAI, former alignment researcher and current board member Paul Christiano has also questioned whether the AI industry is currently moving quickly enough to reduce the risk of catastrophic loss of control.
Christiano said he did not believe the industry, including OpenAI, was currently on track to reduce the risk to what he considered an acceptable level.
These comments demonstrate that the debate is no longer restricted to a single researcher or company.
Anthropic Responds With More Safety Work
While researchers have been raising concerns, Anthropic has continued to invest heavily in AI safety and evaluation.
The company has also taken steps to increase independent evaluation of its models.
On September 18, Anthropic announced a partnership with Accenture under which the two companies each expect to invest at least $1 billion over five years in AI safety work.
Accenture’s specialist AI business, Faculty, will help evaluate and red-team Anthropic’s models, conduct alignment assessments and test safeguards. The evaluators are expected to have access inside Anthropic comparable to that of employees.
A Debate That Is Still Far From Settled
The warnings from Coxon and other researchers have intensified an already heated debate over the future of artificial intelligence.
Some researchers believe the risks deserve urgent attention because future AI systems could become much more capable than today’s models.
Others argue that some of the most extreme predictions remain speculative and that the industry should focus on measurable risks such as cyberattacks, biological misuse, fraud, misinformation and autonomous systems rather than treating human extinction as an established outcome.
What is clear is that the rapid development of frontier AI has created a serious debate inside the companies building the technology themselves.
For Coxon, the central message is that developers should not wait until AI becomes too powerful to control before putting stronger safety measures in place.
The debate over how quickly AI should advance, how much oversight companies should accept and how much risk society should tolerate is therefore likely to remain one of the biggest technology discussions of the coming years.
