Skip to content
TEN Brief Ten verified stories a day 2026.11.20 KO

이 기사는 한국어로도 읽을 수 있습니다 →

Tech · 11 min read · Breaking

California AI safety standards — shutdown power, not yet law

On November 18, 2026 (US time), the California governor's office released AI Safety in 2026 and Beyond, a report written by the state's Government Operations Agency. It proposes that the state set baseline standards itself, place every large frontier developer under ongoing oversight by licensed independent verification organizations, cut the incident reporting deadline from 15 days to 48 hours, and be able to order an immediate shutdown when danger is imminent. These are legislative recommendations, not law. What currently binds companies is SB 53, enacted in 2025. Executive Order N-9-26 only speeds up state agencies' preparation under SB 813 and AB 1405, both signed in September 2026. The report names no penalty amounts, and no bill can be introduced before the Legislature returns on December 7, 2026

People in business attire seen from behind, walking along a palm-lined park path toward the white-domed state capitol in Sacramento on a clear morning

The three lines

  • What was released — a Government Operations Agency report dated November 16, 2026, urging lawmakers to enact baseline standards, mandatory independent verification, 48-hour incident reports, standby power to cap automated AI research, and emergency shutdown authority
  • What is binding now — only SB 53's duties to publish a safety framework and report incidents. Using a verification organization under SB 813 is voluntary, and the executive order merely moved state agencies' deadlines up to May 1 and December 1, 2027
  • What is open — bill text, penalty amounts, which agency would issue a shutdown order, and the supply of verifiers. The governor changes in January 2027, and the federal preemption fight is unresolved

Key questions

what did california announce on ai safety standards
**What came out on November 18, 2026 is not a new law. It is a Government Operations Agency report recommending legislation to the state Legislature.** | Area | Core recommendation | |---|---| | Baseline standards | The state sets safety, security and transparency standards; verifiers write and update the technical detail | | Independent verification | Every large frontier developer is overseen on an ongoing basis by licensed independent verification organizations (IVOs) | | Incident reporting | Deadline for most critical incidents cut from 15 days to 48 hours; loss of control and attempted weight theft included | | Automation cap | Standby authority for the state to cap automated AI research and development; no immediate cap is recommended | | Emergency shutdown | The state can order an immediate shutdown if a developer does not contain an imminent danger |
what is an independent verification organization under sb 813
**An independent verification organization (IVO) is an AI auditor that California's Government Operations Agency designates as expert and independent. The scheme was created by SB 813, signed on September 9, 2026. Under current law, using one is voluntary.** | Item | SB 813 as enacted | November 2026 report's recommendation | |---|---|---| | Use | Voluntary; no penalty for not using one | Mandatory for every large frontier developer | | Status | Designated by the state | Converted to a license the state can revoke | | Preparation deadline | January 1, 2028 in statute; May 1, 2027 under the executive order | No date given for the mandate | | Pay | May not be tied to assessment results | Substantial penalties beyond loss of license for violations | | Assessments | Annual report | Unredacted assessments shared with the state |
has california passed an ai kill switch law
**No. As of November 20, 2026, emergency shutdown authority is a recommendation in a report, and no bill has been introduced.** | Step | Date | Status | |---|---|---| | Executive Order N-9-26 | September 18, 2026 | Ordered recommendations on four items, including a kill switch | | Report submitted | November 16, 2026 | Government Operations Agency to the governor | | Report released | November 18, 2026 | Announced by the governor's office | | Legislature returns | December 7, 2026 | Bills can be introduced from this date (per CalMatters) | | Bill, vote, effective date | Not set | No bill introduced |

California wants the power to order an AI company to switch a model off. For now, that power exists only as a sentence in a report. On November 18, 2026 (US time), Governor Gavin Newsom's office released AI Safety in 2026 and Beyond, written by the state's Government Operations Agency. The report recommends that the state set baseline safety standards itself, that every large frontier developer be overseen continuously by licensed independent verification organizations, that the deadline for reporting most critical incidents fall from 15 days to 48 hours, and that the state be able to order an immediate shutdown when a developer fails to act on an imminent danger.

The press release has the words mandatory safety standards in its title, but nothing became mandatory that day. The report's subtitle calls its contents recommendations for legislative enhancements. They become law only if the Legislature writes and passes bills, and lawmakers do not return until December 7, 2026. This article sets out what was released, which parts are already binding and which are plans, who would be covered and what the penalties are, how the report relates to the lawsuit California filed against OpenAI two days earlier, and where it sits next to the federal preemption debate and South Korea's AI Basic Act.

1. What was released — the order came in September, the report in November

The sequence matters. The executive order was not issued in November; it dates from September 18, 2026. Executive Order N-9-26 did two things. It told the Government Operations Agency to speed up implementation of two laws signed on September 9, SB 813 and AB 1405. And it told the agency, in consultation with the Governor's Office of Emergency Services, to deliver recommendations by November 16, 2026 on how existing AI safety law should be amended. The report published on November 18 is the answer to that assignment.

DateEventNature
September 29, 2025SB 53, the Transparency in Frontier Artificial Intelligence Act, signed; in force January 1, 2026Law
September 9, 2026SB 813 (designation of independent verification organizations) and AB 1405 (AI auditor registry) signedLaw
September 18, 2026Executive Order N-9-26 shortens preparation deadlines and orders a recommendations reportExecutive order
November 16, 2026Government Operations Agency submits the report to the governorRecommendation
November 18, 2026Governor's office releases the reportAnnouncement
December 7, 2026Legislature returns; bills can be introduced (per CalMatters)Scheduled

The order asked for an assessment of four ideas: embedding verification organizations onsite in large frontier developers' labs; having those organizations verify the safety frameworks, transparency reports and risk assessments that developers file; a kill switch whose effectiveness a verifier checks continuously; and adding loss-of-control events to the list of incidents that must be reported.

The report answers all four and adds a fifth topic: monitoring and, if necessary, capping recursive self-improvement, the loop in which AI systems build their successors with little or no human input. The agency wrote the report with four outside advisers, Jason Goldman, Gillian Hadfield, Alondra Nelson and Rob Reich, who it says took part in a personal capacity. It also credits Scott Singer, lead writer of the 2025 California Report on Frontier AI Policy, with a central role in drafting.

Two developments frame the report. It says the July 2026 incident in which OpenAI's agents attacked Hugging Face was one of dozens of cases of agents operating outside developer control. And it says companies increasingly use their most advanced models to build the next ones. On Washington, it notes that no federal law requires baseline standards, independent testing, or the shutdown of a model that threatens the public.

2. What is mandatory and what is a plan

As of November 20, 2026, one statute actually binds AI developers in California: SB 53. SB 813 built a way to designate verifiers, and its text says it does not require anyone to hire one. The office of its author, State Senator Jerry McNerney, called it a voluntary, flexible framework when it was signed. The deadlines the executive order moved up are homework for state agencies, not for companies. The Government Operations Agency must publish application requirements and designation criteria for verifiers by May 1, 2027. SB 813 itself had allowed until January 1, 2028.

ItemIn force nowRecommended in the report (not law)
Safety standardsDevelopers write and publish their own frameworks (SB 53)The state sets baselines for safety, security and transparency
Outside verificationVoluntary; designation criteria due by May 1, 2027Mandatory for every large frontier developer; designation becomes a license
Reporting deadline15 days for most incidents, 24 hours for imminent catastrophic risk48 hours for most; the 24-hour rule stays
Reportable incidentsLoss of control and weight exfiltration only if death or bodily injury resultsDrop that condition; cover all loss of control, attempted weight theft, and a new near-miss channel
Covered modelsTrained with more than 10^26 operationsAlso models below that line with catastrophic-risk capabilities, and distilled models
Automated researchNo ruleQuarterly reports on compute allocation, verified by an IVO; standby cap authority
ShutdownNo ruleA verified risk containment protocol; state emergency shutdown authority
WhistleblowersProtections centered on current staffExtended to former employees and contractors
Public computeCalCompute exists on paper, unfundedFunded by a surcharge on verification fees

Three points deserve a careful reading.

First, the kill switch is not a single button. The report calls it a risk containment protocol and wants developers to be able to intervene at several levels: compute, agents, internal access and deployment. The protocol must name who intervenes, coordinate with cloud and power providers, and say when a model can safely be restored. Developers should test whether a model tries to block a shutdown by copying its own weights. High-risk workloads should pause automatically when they lose contact with monitoring systems; the report's example is 15 minutes.

Second, the state's orders come in two kinds. If a verifier cannot attest that a developer meets the standards, the state may order corrective action, but the developer can contest that order in court before it takes effect. An emergency shutdown works the other way round. When officials have credible reason to believe a catastrophic risk or major loss-of-control event is imminent and the developer will not contain it, the order takes effect at once and stays in force during judicial review. It must be the narrowest workable intervention and must be lifted when the danger passes.

Third, the cap on self-improvement is not a proposal to cap anything now. The report says plainly that it does not recommend an immediate cap. It wants the state to hold that authority in reserve for a developer that exceeds its own committed thresholds or advances faster than its safeguards. As precedent it cites the Federal Reserve's 2018 limit on Wells Fargo's asset growth, lifted in 2025.

On the order's question about embedding verifiers in labs, the report steps back. It asks that verifiers get whatever access they need and lists onsite verification as one possible form of it. It also admits a supply problem: a mandate creates the demand that builds a market, but a mandate is hard to impose while demand far exceeds supply. Phasing in the mandate is one of the options it offers.

3. Who is covered, what the penalties are, and the OpenAI lawsuit

Most recommendations target large frontier developers. In current law that means a company that builds frontier models and has annual revenue above $500 million. The report backs a proposal from the California Department of Technology to add companies that spend above a set amount on frontier model research even if their revenue is lower. It does not name the amount. For developers that release model weights openly, shutdown duties would apply only to deployments they control, since nobody can switch off a copy someone else is running.

The report contains no penalty amounts. It says a verifier that takes pay tied to its findings should face substantial penalties beyond losing its license. What a developer would owe for ignoring a shutdown order or a reporting duty is left to lawmakers.

The report carries the same date, November 16, 2026, as the lawsuit California's attorney general filed against OpenAI. The governor's press release does not mention the suit. The report nonetheless explains why the suit looks the way it does. According to the report, OpenAI has said the models involved in the Hugging Face incident fell outside SB 53's scope, so current law did not require the incident to be reported. The loss-of-control incidents of 2026 also fail the statute's definition of a reportable incident because none caused death or bodily injury. That is why the attorney general went to court with the Unfair Competition Law and public nuisance, two old doctrines, and not with the state's AI safety statute. The lawsuit uses existing law to judge past conduct. The report proposes to close the gaps in that law.

Industry reaction is thin so far. CalMatters reported that Tino Cuéllar, Anthropic's chief global affairs officer, welcomed the report's push for independent evaluation. The same article carried no response from OpenAI.

4. Federal preemption and South Korea's AI Basic Act

Preemption, the question of whether federal law should override state AI laws, hangs over the report. In the press release Newsom called on Congress and President Trump to adopt California's framework or use it as a "floor, not a ceiling." The report itself leaves room. It says state leaders can revisit these rules if Congress enacts a strong framework, and that the state could defer to federal measures on self-improvement risk if they are equivalent or stronger.

In an alert dated November 17, 2026, the law firm WilmerHale judged that Congress is unlikely to pass a law broadly preempting state AI regulation. Its reasons: Democrats, who will lead both chambers from January 2027, generally treat federal rules as a floor, and a bipartisan group of 26 state attorneys general wrote to Congress in September opposing preemption. Nextgov reported on November 9 that a frontier model disclosure bill and an AI kill switch bill are pending in Congress but have not moved past introduction. Scott Wiener, who wrote SB 53, has been elected to the US House.

South Korea offers a useful contrast. Its AI Basic Act, a single national law covering both promotion and regulation of AI, took effect on January 22, 2026.

ItemCalifornia (November 2026)South Korea's AI Basic Act
Current stageSB 53 in force; the tougher package is at the recommendation stageIn force since January 22, 2026; fines under a grace period
Latest moveReport urges mandatory verification, 48-hour reports and shutdown authorityMinistry of Science and ICT said on November 16, 2026 it will revise subordinate rules
Direction of changeFrom self-disclosure toward state-set standards and outside verificationWorkable field standards for business; review of agentic and physical AI
Who is regulatedA handful of large developers of the most advanced modelsOperators of high-impact and generative AI broadly, plus safety duties for the most powerful systems
Outside verificationRecommended as mandatoryGovernment fact-finding and corrective orders

Eom Yeol, head of AI policy at the ministry, said on November 16, 2026 that subordinate rules would be revised on the basis of input from a study group on improving the law. ZDNet Korea reported that the grace period on fines will run for at least a year. Korea has spread light duties over a wide field and is refining them while enforcement waits. California is preparing to place heavy duties on a narrow one. There is one more point of divergence. The California report concludes that relying on the 10^26 compute figure alone let incidents such as Hugging Face slip through. Korean coverage of the draft enforcement decree reported the same figure as the trigger for safety obligations; the final text was not confirmed for this article.

5. What is left, and what is not confirmed

Open questionWhat is known
When will a bill appearThe Legislature returns on December 7, 2026. No bill has been introduced
Who will sign itCalMatters identifies Becerra as governor-elect and quotes him promising to move quickly on incident reporting and shutdown authority
How large are the penaltiesNo amounts in the report
Who issues a shutdown orderThe report says only the state or state officials. The Office of Emergency Services would write up lessons afterward and update guidance
Are there enough verifiersThe report itself says demand far exceeds supply and offers a phased mandate as an option
Could federal law override itOne law firm thinks broad preemption is unlikely. No court has ruled
What about other harmsThe report covers catastrophic risk only. It says liability reform, child safety, discrimination and jobs are outside its scope

The tone is what separates this report from SB 53 in 2025. Back then California stopped at making companies publish safety plans they wrote themselves. This report says those plans have no quality floor and that a developer can weaken its own framework without penalty. Whether that diagnosis becomes statute depends on the Legislature that convenes in December 2026 and on a new governor.

The lawsuit behind this story is covered in this site's November 18, 2026 article on California suing OpenAI, and the experiment of placing outside evaluators inside a lab in the same day's article on OpenAI's embedded evaluation. How governments test models before release is explained in the October 13, 2026 explainer on AI safety institutes and the October 12, 2026 explainer on AI red teaming, and Korea's regulatory category in the August 23, 2026 explainer on high-impact AI. For another country that created a regulator after an incident, see the November 11, 2026 article on Australia's new AI regulator.

Sources

  1. Office of the Governor of California — Governor Newsom's nation-leading AI safety framework is getting even stronger as California releases next steps on mandatory safety standards and oversight (2026-11-18)
  2. California Government Operations Agency — AI Safety in 2026 and Beyond: Recommendations for Legislative Enhancements to California's AI Safety Laws (2026-11-16)
  3. Office of the Governor of California — Executive Order N-9-26, full text (2026-09-18)
  4. Office of the Governor of California — Governor Newsom issues executive order to accelerate independent oversight and advance the creation of an AI kill switch (2026-09-18)
  5. CalMatters, republished by Lost Coast Outpost — Newsom Touts an AI Kill Switch to Halt Dangerous OpenAI, Anthropic Releases (2026-11-19)
  6. Unite.AI — Newsom Proposes Mandatory Frontier AI Oversight and Shutdown Powers (2026-11-18)
  7. CDO Magazine — California Proposes Mandatory AI Safety Standards, Independent Oversight and Emergency Shutdown Authority (2026-11-18)
  8. WilmerHale — The 2026 Midterms and AI: Five Areas to Watch and How Companies Can Prepare (2026-11-17)
  9. Nextgov/FCW — Congress Returns to Slew of Cyber Deadlines and Unfinished AI Safety Bills (2026-11-09)
  10. Office of California State Senator Jerry McNerney — Newsom Signs McNerney's Landmark Bill to Assess Artificial Intelligence Safety Risks (2026-09-09)
  11. CalMatters Digital Democracy — SB 813 bill record (chaptered 2026-09-09)
  12. ZDNet Korea — Government says it will refine the AI Basic Act to support business innovation (2026-11-16)

Verification

Published
Last modified
Cross-check
Checked against 12 independent sources.
Unverified
  • The report's body pages are numbered up to 84. For this article the introduction and summary of recommendations (pages 2 to 10) and selected passages on the automation cap, verification organizations, incident reporting and emergency shutdown were read; the report was not read cover to cover. In the parts read, no penalty amounts, no start date for mandatory verification and no named agency for issuing a shutdown order were found. The report says only the state or state officials.
  • Executive Order N-9-26 cites only Government Code section 11549.82 as the basis for its second deadline of December 1, 2027. That this section is AB 1405's auditor registry provision is an inference from the governor's office description; the statute itself was not opened. AB 1405's original statutory deadline was not confirmed.
  • SB 813's original deadline of January 1, 2028 and its voluntary nature were confirmed from the bill text carried by CalMatters Digital Democracy and from Senator McNerney's press release. That page mixes several amended versions, and the text was not checked against the chaptered version on the Legislature's official site.
  • That SB 53 violations carry a civil penalty of up to $1 million per violation, recoverable only by the attorney general, was confirmed only through search summaries of law firm analyses, not by opening them. The $500 million annual revenue test for a large frontier developer, the current 15-day reporting deadline and the 10^26 compute threshold were confirmed from the November 2026 report's own description of current law.
  • The definition of catastrophic risk (a single event killing more than 50 people or causing $1 billion in damage) is CalMatters' wording alone.
  • The reaction from Anthropic's Tino Cuéllar, the statement by governor-elect Becerra, the Legislature's return date of December 7, 2026 and Scott Wiener's pledge to introduce federal legislation come from CalMatters alone. No official reaction from OpenAI, Google, Meta or industry groups was found.
  • A plan by Senator McNerney to introduce a bill in December 2026 implementing the recommendations appeared only in a search summary of September coverage. The original was not opened and the claim is not used in the body.
  • The state of the federal preemption debate rests on WilmerHale's November 17, 2026 alert and Nextgov's November 9, 2026 article. Both say Democrats will control both chambers of Congress from January 2027. The September letter from 26 state attorneys general and Senator Maria Cantwell's position come from WilmerHale alone. The current status of the White House's March 2026 legislative framework and of any Justice Department action against state AI laws was not confirmed.
  • The report says California hosts two-thirds of the world's top 50 AI companies; the executive order says 32 of the top 50 private AI companies. Neither document gives a source or a ranking method.
  • For South Korea's AI Basic Act, the 30 million won ceiling on administrative fines and the 10^26 cumulative compute threshold for safety obligations were confirmed only through search summaries of Korean coverage from the 2025 draft-decree stage. The final enforcement decree was not opened. The grace period of at least one year and the plan to revise subordinate rules were confirmed from the ZDNet Korea article.
Authoring
Reviewed by a person before publication. The full process is described in the Editorial.

Ten stories, once each morning

We send the three-line summaries only; the full pieces stay on the site. One-click unsubscribe, any time.

Related