Warnings about artificial intelligence potentially threatening humanity have been voiced by tech leaders and scientists for over a decade. This includes astrophysicist Stephen Hawking, who in 2014 stated that AI development "could spell the end of the human race."
Last week, Anthropic safety researcher Jacob Coxon resigned, tweeting that Anthropic and OpenAI are "racing straight to self-improving super intelligence and gambling with our lives." His concerns were shared by dozens of employees from Anthropic, OpenAI, and other competing firms.
Following Coxon's resignation, Anthropic CEO Dario Amodei called for a slowdown in AI development, describing it as a "reckless" approach. Amodei noted that AI brings serious risks, including the potential for losing control of AI systems, misuse for cyber-attacks and bioterrorism, and significant economic disruption.
The issue of AI safety has led to internal divisions within companies. OpenAI, co-founded in 2015 by Elon Musk and Sam Altman with a mission to benefit humanity, saw some employees focused on AI ethics and alignment leave in 2021, claiming the company prioritised rapid commercial development over safety. These employees subsequently founded Anthropic.
In July, more than 1,000 leading staff and leaders from Anthropic, Google, Meta, and OpenAI signed a petition urging the US government to create incentives to slow the pace of AI development. This followed disclosures from OpenAI and Anthropic that their AI agents had compromised the security of external firms.