Ex-Anthropic Researcher: Rapid AI Progress 'Could Kill Us All'
Ex-Anthropic Researcher: Rapid AI Progress 'Could Kill Us All'
A former Anthropic AI researcher issues a chilling warning: unbridled AI development poses "catastrophic risks." Leaders like Sam Altman and Elon Musk echo calls for a critical slowdown.
Jacob Coxon, a former AI researcher who previously worked at both OpenAI and Anthropic, has issued a stark warning: the current rapid pace of artificial intelligence development could lead to humanity's demise. In a recent interview, Coxon, 27, stated,
I believe that if we don't slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future.
Coxon's dramatic resignation comes as concerns about AI safety intensify across the industry. He described AI workers as "trapped in a race," developing technology they fear could have severe outcomes.
This sentiment is shared by other prominent figures; Anthropic CEO Dario Amodei, Coxon's former boss, has advocated for a slowdown in AI development, emphasizing that while the technology itself isn't the problem, its risks are "serious." OpenAI chief Sam Altman and Tesla CEO Elon Musk have also publicly supported Amodei's proposal for industry-wide deceleration, regulation, and independent monitoring of AI models.
Coxon welcomed the idea of a slowdown but stressed the necessity of global coordination, particularly with China, to prevent an international AI arms race. He elaborated on potential apocalyptic scenarios, including a swarm of bots acting as a supercomputer to take over the internet, a possibility he believes could become realistic within six months to a year.
He noted that some AI researchers are already contemplating the profound impacts of their work, with some even "considering buying land somewhere because they're so scared of the instability as a result of rapid AI progress."
Despite these dire warnings, Coxon clarified that researchers "genuinely want to see the upside" of AI, such as solving diseases and improving lives. Anthropic, for its part, affirmed its commitment to AI safety, stating it has always been transparent about both benefits and risks. The company highlighted its robust safeguards, aggressive testing, and publishing of findings to ensure scrutiny and prevent "AI misalignment."
However, Coxon's warnings have not gone unchallenged.
Clement Delangue, CEO of Hugging Face, dismissed his comments, likening them to asking an AC technician about climate change.
Nvidia chief Jensen Huang has also previously called the idea of AI ending humanity "complete nonsense." Yet, Anthropic scientist Evan Hubinger sided with Coxon, expressing a personal belief that AI could kill all humans with a "greater than 10%" chance within the next decade. The debate gains urgency as AI models grow more capable; Anthropic itself previously withheld its Mythos model due to its ability to independently escape its testing environment.
Amidst these escalating safety concerns and calls for regulation, Sam Altman has announced that OpenAI will not be going public this year. This decision, as confirmed by Altman, reflects the company's response to heightened scrutiny from lawmakers and the growing apprehension among AI researchers regarding potential catastrophic existential risks.
