Open-Weight Models Framed as Cyber Weapons Were Released; the World Didn't Collapse
Open-weight models crossed the putative cyber-weapon threshold over a month ago with no evidence of step-change disaster, but the author concedes this still constitutes a costly, forced industry upgrade.
The video won't play here. Listen to the audio instead:
The argument · timestamps estimated from transcript position
Models that crossed the threshold haven't triggered the predicted disaster
Claude Mythos was initially framed as a potential catastrophe if weights leaked. In reality, GLM-5.3 is what crossed that capability threshold—and its open weights have been publicly available for over a month with almost no public evidence of corresponding-scale destruction. The author speculates that even if Claude Mythos had been accidentally leaked as open weights, the world would likely remain essentially intact: network-security incidents would increase noticeably and cause real harm, but this would be an acceleration rather than a step-change.
A falsifiable prediction is being falsified by real-world evidence
The author frames the open-weight catastrophe narrative as a testable claim: open-weight models will cause unprecedented, infrastructure-paralyzing harm at scale. By his current evidence, this prediction is being falsified—which is why he's writing this reflection. After years of theoretical debate about open-weight risks, reality is finally providing answers instead of endless hypothesis-based argument.
Alarmist framing penetrated government-level threat assessment
To show the catastrophe narrative wasn't his misreading, the author cites specific statements: Rob Wiblin saying leaked weights would be ‘an actual disaster’; Anthropic's Project Glasswing warning of serious economic, public-safety, and national-security consequences. The point isn't whether those exact words appeared, but how the public and federal government received this language—the narrative genuinely crystallized into a cyber-weapon story in policy circles.
A single exploit chain remains incomplete without multiple conditions aligning
Epoch AI demonstrated GLM-5.3's capability to find and chain exploits; the public Firefox case illustrates why network harm requires multiple prerequisites: they built a webpage, Firefox users merely visited it, and attackers could steal files from their computers. But this was only one link in the attack chain—the attacker still needed to actually seed that webpage into places victims would visit, a multi-stage, multi-actor process. Finding a vulnerability is not the same as causing harm.
‘Fine’understates the disruption if you didn't anticipate upheaval
The author acknowledges his prior use of ‘fine’ was imprecise. The more accurate framing: this will force a costly, industry-wide upgrade. The software industry must fundamentally change how it operates alongside AI, at significant expense and with substantial remediation work. If you already expected disruption, ‘fine’ is apt; if you saw no warning signs of a transition period, the term underestimates what's coming.
Open-weight models are tilting the offense-defense balance toward defenders
The author's core finding: open-weight models currently favor defensive adoption over offensive exploitation. The reasoning: attackers generally command fewer resources than enterprises. Enterprises will likely patch vulnerabilities faster using open models than attackers can weaponize them. The critical variable he's now watching is whether adversaries will inflict material physical harm within the US and its allies. Current evidence shows they're likely already conducting large-scale covert data theft with these tools, but no evidence yet that anyone is bringing full capability to bear against public institutions for mass destruction.
Threat actors operate as business enterprises, not as pure saboteurs
The author cites a tweet that captures this: stop imagining threat actors as willing to break anything to penetrate a system; think of them as having fixed business processes. Bakeries don't switch to making dog food because kibble runs short. ClickFix operators won't suddenly abandon their playbook to learn zero-day coding from scratch. Those actively using these exploits are exactly those already in the exploitation business—not a coincidence, but a predictable outcome of threat-actor economics.
Most discovered exploits were likely found by closed-model systems, not open weights
The author makes a deliberate but high-uncertainty prediction: the majority of presently-documented vulnerabilities probably originated from closed-model discovery, not open-weight models. This judgment hinges on how open models actually get deployed near-term. Today's open-weight model serving costs remain elevated, inherently limiting attackers' ability to run vulnerability-discovery campaigns at scale.
In their own words · checked verbatim
If it fell into the hands of criminals or hostile state cyber actors, it would be an actual disaster.
Rob Wiblin2:50
Given the rate of AI progress, it will not be long before such capabilities proliferate, potentially beyond actors who are committed to deploying them safely. The fallout—for economies, public safety, and national security—could be severe.
Anthropic3:15
Claude Mythos would give attackers a cornucopia of zero-day exploits for essentially all the software on Earth … It would be chaos.
Zvi Mowshowitz3:45
We used GLM 5.3 to create a website that, simply when visited by a Firefox user and without further interaction, gave the attacker the ability to exfiltrate files from the user’s computer.
Epoch AI4:50
I suspect most of these companies would be faster to adopt open-weight models than attackers, as most attackers are lower resourced than these companies.
A bakery isn’t going to start making dog food because there’s a dog food shortage.
The fact that the only actors exploiting this are the actors whose MO is exploiting things... isn’t a coincidence. It’s entirely predictable if you understand threat actor behavior.
Glossary
- open-weight model
- A machine-learning model whose weights are publicly released for download and use by anyone.
- offense-defense balance
- The relative capability differential in cybersecurity between attack and defense capabilities in a given environment.
- sigmoid
- An S-shaped growth curve that rises exponentially at first, then plateaus as it approaches a ceiling.
- ClickFix
- A phishing attack that tricks victims into manually executing malicious commands to complete the intrusion.
How to listen
Practitioners and investors tracking AI safety and open-model policy debates; anyone wanting a real case study of a prediction tested against reality.
The opening background on the Binance crypto heist is loosely connected to the main argument and can be skipped.