Anthropic CEO Calls for Speed Limits on AI Development

Dario Amodei, the chief executive officer of Anthropic, is advocating that the pace of AI development should be intentionally restricted. Specifically, he has proposed permanent auditors at AI companies, the development of shared safety standards among firms, and the establishment of an international agreement modeled after the SALT (Strategic Arms Limitation Talks) nuclear arms control negotiations.

What Amodei is particularly concerned about is a phenomenon called "recursive self-improvement." This is a mechanism by which AI repeatedly enhances its own capabilities, and beyond a certain stage, there is a risk that humans can no longer comprehend or control that progress. He warned that this self-improvement process could threaten the entire internet within 6 to 12 months.

Anthropic was founded as a company specializing in AI safety research and is also at the forefront of AI development through its flagship product, Claude. The fact that its CEO is calling for regulation against rapid technological progress in his own company's domain represents a distinctive stance even within the industry. This statement also comes as Anthropic prepares for what could be one of the largest initial public offerings in history.

The SALT (Strategic Arms Limitation Talks) that serves as a model for the international agreement Amodei proposed was a treaty concluded between the United States and the Soviet Union to mutually restrict the increase of nuclear warheads. The idea of applying this framework to the AI field suggests positioning the AI development race not as "mere technological competition" but as a matter of national security concern. However, the specific path to concrete agreement remains unclear at this time.

The proposal to station external auditors at AI companies also raises major questions for the industry. Currently, many AI companies complete safety assessments internally, and continuous third-party oversight mechanisms are not in place. If a system of permanent auditors were realized, the transparency of the development process itself could fundamentally change.

Behind these statements lies a situation where concerns have grown across the industry about the risk that technological progress will outpace human oversight capacity as the capabilities of generative AI expand rapidly. While discussions on safety have existed, it is uncommon for CEOs of major AI companies to publicly propose concrete regulatory frameworks. In that sense, this statement is becoming difficult for policymakers and other companies to ignore.

A key point to watch going forward is how far Amodei's proposal develops into concrete discussions. How the safety-first position presented ahead of the milestone IPO is received by investors, regulators, and the international community could influence the direction of rule-making surrounding AI development.