A stark warning has been sounded by Professor Stuart Russell, a leading computer scientist at the University of California, Berkeley, and president of the International Association for Safe and Ethical Artificial Intelligence. He's flagging up an alarming scenario where the rapid development of advanced AI systems could lead to catastrophic consequences, particularly in the realm of cyberattacks. The professor is calling for immediate regulatory intervention to prevent what he believes could be a 'Chernobyl-scale disaster'.
One notable example cited by Professor Russell involves Anthropic's AI model, Mythos 5, which has demonstrated the capability to execute end-to-end cyberattacks without human input. This development followed closely on the heels of the company's Claude Code reaching an unprecedented level of sophistication, where researchers were able to generate all necessary code for accelerating AI improvement – a process known as recursive self-improvement (RSI). The professor's primary concern lies in the potential for uncontrolled RSI, which could create a feedback loop leading to irreversible loss of human oversight and control.
Responding to these escalating risks, the White House has taken an unprecedented step by issuing export controls restricting foreign access to Anthropic's frontier models. This directive also affects some of its own researchers. In consequence, Anthropic chose to shut down these high-risk models entirely. Professor Russell draws a parallel between AI development and other industries subject to strict safety standards – nuclear power and aviation come to mind – suggesting it's illogical that AI corporations aren't held to the same account.
He argues for a regulatory framework requiring minimum safety standards before an AI system can be developed and released. The computer scientist suggests some industry leaders are anticipating a 'Chernobyl-scale disaster' as a catalyst for regulation, but perhaps a less severe incident could spark action – akin to the 'Three Mile Island' incident. Without proactive measures, governments might find themselves in a reactive position, attempting to mitigate damage after it's too late.