California Governor Orders 'Emergency Stop' for AI

California Governor Gavin Newsom has signed an executive order requiring the implementation of an 'emergency stop switch (kill switch)' for AI models. Additionally, the order requires the placement of independent auditors within AI companies, clearly demonstrating the state government's commitment to establishing an AI safety management framework.

This measure is driven by the reality that federal-level AI regulation remains insufficient. Governor Newsom points out that there currently is no federal law mandating AI companies to report dangerous incidents. In other words, even if AI creates a serious risk, companies face no legal obligation at the federal level to disclose such information. The primary aim of this action is to address this regulatory gap through the state's own executive order.

The executive order's concrete provisions fall into two main categories. One is the placement of independent auditors within AI companies. The other is the requirement to implement a 'kill switch' that allows forced shutdown of AI models when problems arise. Furthermore, an expert panel has been established to develop specific safety recommendations, with submissions required within two months.

A 'kill switch' refers to a mechanism to quickly halt or control an AI system when it behaves unexpectedly or when serious risks are identified. As AI technology spreads rapidly, ensuring humans can reliably regain control in worst-case scenarios has become an emphasized principle in international safety discussions. This executive order seeks to translate this principle into state policy, raising the question of how to design an effective safety net.

California is a region where major AI companies like OpenAI and Google are concentrated, making the state's policies likely to influence the entire AI industry. This type of state-level regulatory measure represents an instance of individual states establishing safety standards ahead of time, given that the federal government has not yet implemented comprehensive AI legislation.

The content of the recommendations submitted by the expert panel within two months will become a critical turning point in actual policy design. Details such as how to ensure auditor independence and how to establish criteria for activating the kill switch remain undecided. Going forward, attention will focus on the expert panel's recommendations and the state government's subsequent concrete institutional design.

The pattern of states filling federal regulatory gaps is not unique to AI and has been seen historically. However, because AI services are largely delivered across state lines via the internet, a key challenge is how much effectiveness state-level regulation can achieve. A key question for medium to long-term observation is whether this executive order will serve as a model and catalyst for discussion at other states and the federal level.