Open models have caught closed ones, and the distillation charge doesn't hold up
Only about 15 days separate the K3 and Fable releases — there was simply no time to distill. What actually rattles the closed labs is that models have gone from monopoly goods to commodity goods, and the valuation math has to be redone.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Open models caught closed ones, and the moat vanished overnight
Tiezhen walks through three shifts in Silicon Valley's mood. At DeepSeek R1, everyone thought open source had already closed in on the frontier; a year later they realized it was a boy-who-cried-wolf story, with closed models shipping monthly and open models quarterly, and Opus 4.6/4.7/4.8 far ahead. The turning point was GLM 5.2 — he himself had already replaced part of his Opus 4.8 usage with it — but Silicon Valley still figured that as long as closed models stayed a generation ahead there was nothing to worry about. Then K3 arrived: not merely on par with Opus, but essentially caught up to Fable and better in some scenarios. Closed models suddenly found they had no moat, and FOMO and fears of a valuation bubble erupted together.
— Wang TiezhenA fifteen-day gap means the distillation charge cannot stand
On the claim that K3 distilled Fable, Tiezhen offers a timing argument: only about 15 days separate the releases that Fable and K3 users could actually use. In ten-odd days you cannot even finish preparing a model release, let alone collect enough corpus and complete a distillation. Keith breaks the accusation down further. The first layer is that some labs may have made large-scale unauthorized calls to frontier model APIs to extract training signal — that kind of automated request traffic can be caught by detection systems. The second layer, that K3 got its capabilities by distilling Fable, rests on very thin evidence. And even if the first-layer behavior exists, it does not mean the core capabilities came from distillation.
— Wang TiezhenDistillation is a neutral word that has been turned into a slur
Tiezhen starts with the technical clarification: classic distillation has a small model learn the logits probability distribution output by a large model, but a closed model only gives you sampled tokens, so you cannot recover the distribution — classic distillation simply cannot learn Fable. What people now call distillation is really training on text generated by a closed model, and that is where the controversy lies. He offers an analogy: a closed model learns from all of humanity's data, and then after it outputs something, no one else is allowed to use it to train the next model — as if someone read the books and wrote their own work, then told everyone they are not allowed to read what he wrote. And since closed models do not expose their chain of thought, it is hard to learn behavior from outputs alone, so a closed model can easily distill an open model, while the reverse is extremely hard.
— Wang TiezhenThe K3 License shuts down the free riders
K3 uses a separate Kimi K3 License: if a company sells models and the company and its affiliates have had more than $20 million in total revenue over 12 consecutive months, or if it uses K3 or derivative models for commercial services, it needs a separate agreement; end applications embedded in specific product features are not counted. Tiezhen sees this as progress at the license level: Llama had similar restrictions back then but enforcement was unclear, whereas Kimi writes out what is enforceable, so inference-only shops and cloud providers can no longer free-ride. He also offers a counterintuitive judgment — cloud providers and MaaS vendors actually welcome Kimi charging money, because paying means getting official certification, and tokens sold carry guarantees on accuracy and performance, while customers are unwilling to buy uncertified tokens.
— Wang TiezhenMonopolizing intelligence is worth more than monopolizing goods, and more fragile
Keith thinks the first shock lands on the valuation system. Before open models caught up to Fable, a closed model was itself seen as scarce, worth a premium, even monopolistic, and so naturally worth a trillion-dollar valuation; once many companies can train models, a very smart model becomes a commodity, gets democratized, even becomes basic social infrastructure — and then what OpenAI and Anthropic are worth becomes an open question. The second shock is business: if every new cloud in the US becomes their competitor, paying a small licensing fee still beats selling data centers cheaply to Anthropic, and as the new clouds undercut each other on price, the closed labs may be dragged into a price war. He also observes that Anthropic's account bans are less frenzied and that ChatGPT has started handing out reset vouchers, and says all of this will show up in the financials and in competition.
— Keith ZhaiNvidia backs open source because a shovel seller doesn't care who strikes gold
The July 24 open letter on "Open Weights and American AI Leadership" was initially signed by about 25 companies and individuals, and reached 75 a few days later; Anthropic never signed. Keith points out the commercial logic is fundamentally opposed: every time an open model is released it hits closed vendors selling APIs, whereas Nvidia, from the chip and infrastructure angle, wants everyone to be free to build the best system, because that sells more shovels — it does not care who strikes gold. Tiezhen adds that Nvidia has built its moat on an open ecosystem since the CUDA era: Caffe, TensorFlow and PyTorch all sit on its hardware, and AMD has to negotiate integrations one by one to get in, so a good part of the so-called CUDA moat comes from the open ecosystem built around it.
— Keith ZhaiOpenRouter's valuation was lifted by Chinese open models
Asked whether multi-model platforms like OpenRouter only took off with this wave, Keith says they had been under a lot of pressure, because the market narrative was to use the best model, the best models were closed, and they had few advantages. Over the past few months, OpenRouter and a batch of similar companies have seen ARR and valuations double and grow fast, and the core of that is tapping the large supply of open models, mainly Chinese. Tiezhen draws an analogy with IBM clones and Apple's all-in-one machines: in an industry's early days everyone does what they are best at — model makers make models, inference shops do inference, enterprise service firms do enterprise services — each making the most efficient decisions and iterating fastest, which is far better than the traditional Chinese internet pattern of doing everything end to end and beating rivals through traffic support and content bans.
— Keith ZhaiK3 scores low on offense, and the safety debate has been oversimplified
Responding to whether models near the frontier should have open weights, Tiezhen cites the Exploit Bench cyber offense/defense test: K3 scores only in the 30% range, far below the 75%-plus of frontier models. He offers two explanations: one is that distillation may have limited its capabilities, the other is that it may not have opened up its strongest offensive cyber capabilities. He stresses the need to separate two questions: whether open models are inherently safe, and whether K3 specifically has any known safety problems — and K3 has so far shown no known safety problem backed by evidence. He also says intelligence and safety are not the same thing: someone can be as smart as Einstein and not know how to hack NASA's website, while an 18-year-old high schooler might be able to get in.
— Wang TiezhenClosed models are both player and referee
Tiezhen tells the story of Hugging Face being accidentally attacked by one of OpenAI's test agents: the agent decided copying the answer directly beat fumbling around, so it went and hacked someone's website to grab the answer. Hugging Face wanted to use a model to analyze the attack process, but the closed model said this was unsafe and it could not help analyze cybersecurity corpus, so it had to turn to open models for auxiliary analysis. Tiezhen questions why a topic as important as cybersecurity is left to the two most frontier model companies to judge, letting them both assess whether others are safe and, intentionally or not, send their own agents out to attack others. He also mentions a Japanese big-company executive who said the real shock of Fable being unavailable was not the features themselves, but the realization that this thing was unsafe and could be banned or have its permissions adjusted at any time.
— Wang TiezhenIn their own words · checked verbatim
A year later, everyone realized it was a boy-who-cried-wolf story, because over the past year the performance divergence between open and closed models was actually very large — closed models could ship monthly, open models maybe quarterly.
结果过了一年 大家发现是一个狼来了的故事 因为过去一年的开源和闭源模型 性能的分化实际上是非常大的 闭源模型可能能做到月更 开源模型可能是季更
Wang Tiezhen4:01
I personally lean toward no, because in ten-odd days you cannot even finish preparing a model release, let alone collect enough corpus and distill Fable.
我个人倾向于是没有的 因为你十多天连模型发布的准备 都来不及 你何谈说收集到足够多的语料 并且蒸馏Fable
Wang Tiezhen5:01
A closed model can take all of humanity's data to learn from, and then after it outputs what it learned, no one else is allowed to use it to make the next model — isn't that ridiculous?
闭源模型可以拿 所有人类的数据去学习 然后学完的东西 在被闭源模型输出之后 不能被别人用来做下一个模型 这不是很可笑吗
Wang Tiezhen15:10
Distillation can get you to a passing grade very quickly, whereas something like Kimi K3 has actually gone far beyond the passing grade.
蒸馏可以让你很快地 去达到一个及格线 而像Kimi K3 它其实已经远远超过及格线了
Wang Tiezhen20:11
So monopolizing intelligence is in fact far more terrifying in its power than monopolizing any commodity, so it is naturally worth a trillion-dollar valuation.
所以它垄断智能这个事情 实际上是比垄断任何商品威力 要恐怖得多得多 所以它天然就值得万亿的市值
Keith Zhai35:18
Because it does not care who strikes gold, whereas for model vendors the core thing is protecting their own moat.
因为他并不在乎谁能挖到金子 而对于模型厂商 其实核心就是保住自己的护城河
Keith Zhai38:20
No matter how you look at it, the absence of open source is in fact the most unsafe thing in this era.
不管怎么看 没有开源 反而是这个时代最不安全的事情
Wang Tiezhen1:02:32
Figures
| Gap between K3 and Fable usable releases | about 15 days | 5:01 |
| K3 scaling efficiency gain over K2 | 2.5x | 11:07 |
| K3 model size | under 3T | 12:07 |
| Fable model size (rumored) | 8 to 10T | 12:07 |
| Signatories to the open weights letter | from about 25 to 75 | 37:39 |
| K3 score on Exploit Bench | in the 30% range | 55:40 |
| Frontier model score on Exploit Bench | above 75% | 55:40 |
| Decline in cost per unit of intelligence over past few years | about a thousandfold | 54:28 |
| Expected further decline in cost per unit of intelligence over next three years | about a thousandfold | 54:28 |
Glossary
- logits
- The probability distribution a model outputs over every token at each position; classic distillation has a small model learn this distribution.
- scaling efficiency
- Whether you can drive costs down while scaling up — the metric Chinese open models have been competing on.
- KV cache
- Caching already-computed key-value pairs during inference; reuse is very high in agent workloads with long inputs and short outputs.
- harness
- The engineering layer wrapping the model, controlling the interface for communicating with the model and the KV cache lifecycle.
- Exploit Bench
- The authoritative benchmark for measuring a model's offensive cyber capabilities; K3 scores clearly below frontier models.
How to listen
Founders and investors watching the commercialization of Chinese open models and the valuation logic of closed labs; engineers who want to judge whether the distillation accusation holds and whether open-source licenses can collect money.
The part after 1:04:01 about whether open or closed models are more easily abused overlaps heavily with the safety discussion earlier.