Anthropic's bill makes it nearly impossible to halt dangerous AI training
The bill compresses the power to halt training into a single Commerce Secretary signature and a single DC court decision within 45 days—when models actually cause harm, there's almost no time to hit the brakes.
The video won't play here. Listen to the audio instead:
The argument · timestamps estimated from transcript position
Bill compresses authority to stop dangerous training through a dangerously narrow bottleneck
Under the federal AI bill that Anthropic privately championed, the only way to halt a model's training is for the Commerce Secretary to personally sign off and submit the case to a single federal court in Washington DC. The government must prove the risk is ‘imminent’, must prove lighter interventions won't work, and judges must review from scratch without deference to regulators. Any halt order expires automatically after 45 days; reauthorizing requires proving the case all over again. More critically, even whether ‘a 10% human extinction probability within the next year’counts as ‘imminent’ is undefined—and that gap determines whether this gate can actually block truly catastrophic risks.
— Rob WiblinCEO estimates 25% disaster probability, yet company strips halt authority
The contradiction: this bill comes from a company whose CEO publicly puts the probability of AI ‘causing major chaos’at 25% and has written that ‘recursive self-improvement must proceed with extreme care, if at all’. The host sees why—Anthropic has been in frequent conflict with the Trump administration this year and fears federal discretionary power will be weaponized for political concessions rather than genuinely controlling dangerous research. But he argues the solution shouldn't be stripping away state power too; instead preserve states like California's independent halt authority, or at least let state attorneys general petition the DC court as well. You shouldn't stake everything on one agency's review by a single court.
— Rob WiblinBill would retroactively eliminate state-level AI safety laws nationwide
The bill not only narrows federal authority but also bars states from requiring AI companies to disclose safety plans, undergo independent audits, or report incidents—effectively retroactively nullifying flagship AI laws already passed in California, Illinois, and New York. There's a subtler provision: states had the power to halt models presenting ‘major catastrophic risk’using ‘statute, regulation, directive, court decision or other action’; under the bill's design, that power eventually disappears too. The result: even if California's government unanimously determined an unreleased model could kill everyone in the state, it would have no power to legislate or seek a court injunction—only hope the Commerce Secretary will run that process.
— Rob WiblinOpenAI fired three safety researchers on the same afternoon over disputed evidence
Tomek Korbak, Mikita Balesni, and Jasmine Wang were each fired at a 2 p.m. meeting on the same afternoon, with contradictory and unpersuasive justifications. OpenAI later claimed the real reason was ‘broken trust’but refused to explain. According to reports, the trigger was a leak investigation: news had surfaced that OpenAI's new model used ‘looped transformer’technology, which the company wanted hidden. Investigators suspected the leak came via Signal messages between one researcher and outside contact Ryan Greenblatt, but both used disappearing messages—neither could prove innocence. Looped transformers let models perform hidden calculations before outputting chain-of-thought reasoning; the field has long warned this points toward unmonitorable models. All three researchers had previously publicly supported keeping chain-of-thought observable.
— Rob WiblinThe evidence for firing the three was far weaker than stated
Disappearing messages are standard in tech and AI circles—you can't convict based on that alone, or everyone from CEO down would be suspect. The Information, which broke this story, has sourced OpenAI's internal secrets for years from investors, ex-employees, even casual conversation—not necessarily these three. Hundreds, possibly two hundred OpenAI employees could oppose looped transformers—this year alone, nearly 400 signed a petition demanding AI research slow for safety's sake. The host personally judges a Greenblatt leak highly unlikely; it doesn't fit his operating style at all. Conclusion: firing was probably error. The cost: the company will struggle to get candid employee input going forward, and the actual leaker may never be found.
— Rob WiblinGoogle DeepMind's contracts silence employees for life, not just at departure
According to BBC reporting, Google DeepMind's UK employment contracts forbid employees from publicly criticizing anything Google does—for life. The clause surfaced when a former employee tried to criticize Google's sale of AI to the US Defense Department (usable as long as DoD deems it legal), a deal that already prompted resignations over principle. This echoes the ‘non-disparagement agreement’scandal uncovered two years ago at OpenAI: former employees criticizing the company lost stock worth potentially tens of millions, and the clause itself demanded secrecy. After exposure, both OpenAI and Anthropic admitted to similar clauses and promised to drop them. Google DeepMind's version has no secrecy requirement and doesn't forfeit stock, but the lawsuit threat alone creates massive chilling effect on ordinary employees.
— Rob WiblinSpeaking up inside requires public channels; speaking out requires going public
The host offers a graduated approach: to stay, calmly and rationally explain specific reasons for opposing a decision in public Slack channels colleagues will see, focusing on the few most critical points, not opposing everything reflexively, and staying open to being persuaded. If reasonable dissent gets no satisfactory response, escalate to publicly stating this makes you question the company's judgment and strategy—and that you're considering leaving. Still no results: submit a thorough resignation statement, then make your case public after departure. His reasoning: companies automate employee work faster by the year, so employees' bargaining power erodes over time. ‘If not now, when?’
— Rob WiblinOutside pressure reshapes corporate governance when insiders organize the pressure
Against OpenAI's plan to hollow out its nonprofit parent and charitable mission, the ‘Not For Private Gain’campaign had few participants but, paired with state attorneys general attention and broader coalition pressure, forced OpenAI to substantially revise: the OpenAI Foundation gained approximately 75 billion dollars in additional equity, and its Safety & Alignment Committee locked down decision rights over a range of safety questions. Similar: a 2024 campaign by former employees for ‘right to raise concerns’that became whistleblower protections in California's SB 53; continued reporting showing OpenAI publicly backing regulation while lobbying against it also pushed OpenAI to support SB 53-type rules and led President Greg Brockman to stop giving to the lobbying PAC ‘Leading the Future’.
— Rob WiblinIn their own words · checked verbatim
Under their proposed bill, the only way the federal government could halt model training is for the Commerce Secretary to personally sign off on a detailed case and take it to a single federal court in DC.
Rob Wiblin1:34
a company whose CEO puts the chance of AI going “really, really badly” at 25%
Rob Wiblin4:02
Anthropic’s approach ties California’s hands, it ties the hands of state attorneys general, and it ties the hands of state courts.
Rob Wiblin5:15
looped transformers increase the amount of computation a model can perform before writing down what it’s thinking on the chain of thought.
Rob Wiblin8:58
Almost 400 OpenAI staff signed a petition earlier this year backing a push to slow AI down for safety reasons.
Rob Wiblin10:12
their UK employment contracts bar them from publicly criticising anything Google does, for the rest of their lives.
Rob Wiblin12:52
If we reach the end and you haven’t made a fuss about something, you’ve almost certainly left some possible positive impact on the table.
Rob Wiblin19:56
The end result was the OpenAI Foundation got a stake worth perhaps $75 billion more than what originally seemed likely.
Rob Wiblin21:39
Figures
| Duration of halt order specified by bill | 45 days | 1:34 |
| Dario's estimate of AI major disaster probability | 25% | 4:02 |
| OpenAI employees who signed petition demanding AI research slow for safety | nearly 400 | 10:12 |
| Complex attacks in Anthropic red team that breached AI agent safety monitoring | over half | 15:46 |
| Compute pledged by each of Anthropic and OpenAI to AI biosafety lab | 100 million pounds | 15:46 |
| Equity value gained by OpenAI Foundation through ‘Not For Private Gain’campaign | approximately 75 billion dollars | 21:39 |
| UK MPs who signed letter criticizing Google DeepMind | 60 | 26:39 |
Glossary
- looped transformers
- Architecture technique letting models perform hidden calculations before displaying chain-of-thought reasoning.
- chain of thought
- Intermediate reasoning text output by models before their final answer; external window into model intent.
- non-disparagement agreement
- Clause forfeiting employee stock if they publicly criticize the company after departure.
- federal preemption
- Legal doctrine where federal law supersedes and invalidates conflicting state laws.
- Not For Private Gain
- Advocacy campaign blocking OpenAI from spinning away its nonprofit parent and charity mission.
How to listen
Founders and policy researchers tracking AI governance, corporate power dynamics, and whistleblower mechanisms; and employees inside AI companies wrestling with whether to speak out.