Anthropic CEO Dario Amodei has outlined three broad strategies for slowing the pace of AI development in a new blog post, and said the company is unilaterally committing to one of them.
Amodei wrote that two things convinced him it is time for a more cautious approach: the OpenAI-HuggingFace hack, and the fact that AI has been advancing drastically faster in recent months, particularly in its growing ability to build the next generation of AI. "We must slow the pace at which we improve the capabilities of AI models," he wrote. "Progress will still seem fast, and we must make wise use of the time we gain."
His first proposed step involves embedded evaluators from third-party organisations such as METR, who could verify that AI companies are following their pacing and safety commitments and ensure safety incidents are reported. Amodei compared them to regulators embedded with bank employees and said inviting them in is something Anthropic is unilaterally committing to, while calling on governments to require other frontier companies to match. He said this would mean giving evaluators company badges, desks and laptops, and access mostly comparable to internal risk assessment teams, with exceptions when required by law or contracts.
Second, Amodei called for leading AI companies within democratic countries to coordinate common safety standards as well as limits on the rate of unchecked AI progress. He wrote that for antitrust reasons it would be helpful for the US government to mediate or at least enable these discussions, issuing a narrow waiver for certain kinds of safety conversations.
Amodei also acknowledged concerns about Chinese AI dominance, but said that if the US government and tech companies refuse to sell powerful chips or semiconductor manufacturing equipment to Chinese companies and crack down on model distillation, they could slow China's progress enough to widen America's lead significantly over the next three to five years. Third, he called for global coordination, with the United States and its allies attempting to coordinate with authoritarian governments to the extent possible, including cooperation with China. He admitted there are stark limits on what can be achieved but suggested there might be opportunities for agreement, such as prohibiting certain narrow and obviously dangerous uses of AI, including for the production of biological weapons.
The debate over AI safety and alignment intensified this week after researcher Jacob Coxon wrote that he is resigning from Anthropic over concerns that leading AI companies are gambling with our lives. Amodei's post did not explicitly mention the resignation. He wrote that he continues to believe AI can enormously improve the quality of human life, adding: "My desire to achieve these benefits is undimmed. But the benefits will only be achieved if we build the technology in the right way, and — so long as we use the time we gain well — it is worth taking unusually deliberate care to get it right."