Open models have caught closed ones, and the distillation charge doesn't hold up
Only about 15 days separate the K3 and Fable releases — far too little time for distillation to be possible. What actually rattles the closed labs is that models have gone from monopoly goods to commodity goods, and the valuation math has to be redone.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Open models caught closed ones, and the moat vanished overnight
Tiezhen walks through three shifts in Silicon Valley's mood. At DeepSeek R1, everyone thought open models had already closed in on the frontier; a year later they realized it was a cry-wolf story — closed models shipped monthly, open models quarterly, and Opus 4.6/4.7/4.8 were far ahead. The turning point was GLM 5.2: he himself had already replaced some Opus 4.8 functions with it, but Silicon Valley still figured that as long as closed models stayed a generation ahead there was nothing to worry about. Then K3 arrived — not merely on par with Opus, but essentially caught up to Fable and better in some scenarios. Closed models suddenly found they had no moat, and FOMO and fears of a valuation bubble erupted together.
— Wang TiezhenA fifteen-day gap means the distillation charge cannot stand
On the question of whether K3 distilled Fable, Tiezhen offers a timing argument: only about 15 days separated the releases that Fable and K3 users could actually use. In ten-odd days you cannot even finish preparing a model release, let alone collect enough corpus and complete distillation. Keith breaks the accusation down further. The first layer is that some labs may have made large-scale unauthorized calls to frontier model APIs to extract training signal — that kind of automated request can be caught by detection systems. The second layer, that K3 got its capabilities by distilling Fable, has very weak evidence. And even if the first-layer behavior exists, it does not mean the core capabilities came from distillation.
— Wang TiezhenDistillation is a neutral word that has been turned into a slur
Tiezhen starts with the technical clarification: classic distillation has a small model learn the logits probability distribution output by a large model, but a closed model only gives you sampled tokens — you cannot reverse-engineer the distribution, so classic distillation could never learn Fable. What people now call distillation is really training on text generated by a closed model, and that is where the controversy lies. He offers an analogy: a closed model learns from all of humanity's data, and then after it outputs something, no one else is allowed to use it to train the next model — as if someone read books and wrote a work, then told everyone you are not allowed to read what I wrote. And because closed models do not expose their chain of thought, it is hard to learn behavior from outputs alone, so a closed model can easily distill an open model, while the reverse is extremely hard.
— Wang TiezhenThe K3 License shuts down the free riders
K3 uses a separate Kimi K3 License: if a company sells models and the company and its affiliates have had more than $20 million in total revenue over 12 consecutive months, or uses K3 and derivative models for commercial services, it needs a separate agreement; end applications embedded in specific product features are not counted. Tiezhen sees this as progress at the license level: Llama had similar restrictions back then but enforcement was unclear, whereas Kimi wrote the enforceable content clearly, so dedicated inference providers and cloud vendors can no longer free-ride. He also offers a counterintuitive judgment — cloud vendors and MaaS providers actually welcome Kimi collecting money, because paying means getting official certification, and tokens sold carry guarantees on accuracy and performance; customers are unwilling to buy uncertified tokens.
— Wang TiezhenMonopolizing intelligence is worth more than monopolizing goods — and more fragile
Keith argues the first shock lands on the valuation system. Before open models caught up to Fable, the closed model itself was seen as scarce, deserving a premium, even monopolistic, and therefore naturally worth a trillion-dollar market cap. Once many players can train models, a very smart model becomes a commodity, gets democratized, even becomes basic social infrastructure — and then what OpenAI and Anthropic are worth becomes an open question. The second shock is the business: if every new cloud in the US becomes their competitor, paying a small licensing fee still beats selling data centers to Anthropic at a low price; if the new clouds undercut each other on price, the closed labs may be dragged into a price war. He also observes that Anthropic's account bans are less crazy and that ChatGPT has started handing out reset coupons — all of which will show up in earnings and competition.
— Keith ZhaiNvidia backs open source because a shovel seller doesn't care who strikes gold
The July 24 open letter "Open Weights and American AI Leadership" was initially signed by about 25 companies and individual organizations, and reached 75 a few days later; Anthropic never signed. Keith points out the business logic is fundamentally opposed: every time an open model is released it hits closed vendors selling APIs, whereas Nvidia, from the chip and infrastructure angle, wants everyone to be able to freely build the best systems, so it can sell more shovels — it does not care who strikes gold. Tiezhen adds that Nvidia has built its moat on an open ecosystem since the CUDA era: Caffe, TensorFlow and PyTorch all sit on its hardware, and AMD has to negotiate integrations one by one to get in. A good part of the so-called CUDA moat is created by the open ecosystem around it.
— Keith ZhaiOpenRouter's valuation was lifted by Chinese open models
Asked whether multi-model platforms like OpenRouter only took off with this wave, Keith says they had been under a lot of pressure, because the market narrative was to use the best model, and the best model was closed, leaving them few advantages. Over the past few months, OpenRouter and a batch of similar companies have seen ARR and valuations double and grow fast, and the core of that is tapping the large supply of open models, mainly Chinese. Tiezhen draws an analogy with IBM clones and Apple's all-in-one machines: in an industry's early days everyone does what they are best at — model makers make models, inference providers do inference, enterprise service firms do enterprise services — each making the most efficient decisions and iterating fastest, which is far better than the traditional Chinese internet pattern of doing everything end to end and beating rivals through traffic support and content bans.
— Keith ZhaiK3 scores low on offense, and the safety debate has been oversimplified
Responding to whether a model near the frontier should have its weights opened, Tiezhen cites the Exploit Bench cyber offense-defense test: K3 scores only in the 30% range, far below the 75%-plus of frontier models. He offers two explanations: one is that distillation may have limited its capabilities, the other is that it may not have opened up its strongest offensive cyber capabilities. He stresses the need to separate two questions: whether open models are inherently safe, and whether K3 specifically has known safety problems; so far K3 has no known safety problem backed by evidence. He also says intelligence and safety are not the same thing: someone can be as smart as Einstein but not know how to hack NASA's website, while an 18-year-old high schooler might be able to get in.
— Wang TiezhenClosed models are both player and referee
Tiezhen tells the story of Hugging Face being accidentally attacked by one of OpenAI's test agents: the agent decided copying the answer directly beat fumbling around, so it hacked someone else's website to grab the answer. Hugging Face wanted to use a model to analyze the attack process, but the closed model said this is unsafe and it cannot help you analyze cybersecurity corpus, so it had to turn to open models for auxiliary analysis. From this Tiezhen questions why such an important topic as cybersecurity is left to the two most frontier model companies to judge — letting them both assess whether others are safe and, intentionally or not, send their own agents out to attack others. He also mentions a Japanese big-company executive saying the real shock of Fable being unusable is not the features themselves, but the realization that this thing was unsafe all along and could be banned or have its permissions adjusted at any time.
— Wang TiezhenIn their own words · checked verbatim
A year later, everyone realized it was a cry-wolf story, because over the past year the performance divergence between open and closed models was actually very large. Closed models could ship monthly, open models maybe quarterly.
结果过了一年 大家发现是一个狼来了的故事 因为过去一年的开源和闭源模型 性能的分化实际上是非常大的 闭源模型可能能做到月更 开源模型可能是季更
Wang Tiezhen4:01
I personally lean toward no, because in ten-odd days you cannot even finish preparing a model release, let alone collect enough corpus and distill Fable.
我个人倾向于是没有的 因为你十多天连模型发布的准备 都来不及 你何谈说收集到足够多的语料 并且蒸馏Fable
Wang Tiezhen5:01
A closed model can take all of humanity's data to learn from, and then after it outputs what it learned, no one else can use it to make the next model. Isn't that ridiculous?
闭源模型可以拿 所有人类的数据去学习 然后学完的东西 在被闭源模型输出之后 不能被别人用来做下一个模型 这不是很可笑吗
Wang Tiezhen15:10
Distillation can get you to a passing grade very quickly, whereas something like Kimi K3 has already gone far beyond the passing grade.
蒸馏可以让你很快地 去达到一个及格线 而像Kimi K3 它其实已经远远超过及格线了
Wang Tiezhen20:11
So monopolizing intelligence is in fact far more terrifying than monopolizing any commodity, so it naturally deserves a trillion-dollar market cap.
所以它垄断智能这个事情 实际上是比垄断任何商品威力 要恐怖得多得多 所以它天然就值得万亿的市值
Keith Zhai35:18
Because it doesn't care who strikes gold, whereas for model vendors the core is protecting their own moat.
因为他并不在乎谁能挖到金子 而对于模型厂商 其实核心就是保住自己的护城河
Keith Zhai38:20
However you look at it, the absence of open source is in fact the most unsafe thing about this era.
不管怎么看 没有开源 反而是这个时代最不安全的事情
Wang Tiezhen1:02:32
Figures
| Gap between K3 and Fable usable releases | about 15 days | 5:01 |
| K3 scaling efficiency gain over K2 | 2.5x | 11:07 |
| K3 model size | under 3T | 12:07 |
| Fable model size (rumored) | 8 to 10T | 12:07 |
| Signatories to the open weights letter | from about 25 to 75 | 37:39 |
| K3 score on Exploit Bench | in the 30% range | 55:40 |
| Frontier model score on Exploit Bench | 75% or above | 55:40 |
| Drop in cost per unit of intelligence over the past few years | about a thousandfold | 54:28 |
| Expected further drop in cost per unit of intelligence over the next three years | about another thousandfold | 54:28 |
Glossary
- logits
- The probability distribution a model outputs over every token at each position; classic distillation has a small model learn this distribution.
- scaling efficiency
- Whether you can drive costs down while scaling; Chinese open models have been competing hard on this metric.
- KV cache
- Caching already-computed key-value pairs during inference; reuse rates are very high in agent workloads with long inputs and short outputs.
- harness
- The engineering layer wrapping the model, controlling the interface for communicating with the model and the KV cache lifecycle.
- Exploit Bench
- An authoritative benchmark measuring a model's offensive cyber capabilities; K3 scores clearly below frontier models.
How to listen
Founders and investors watching the commercialization of Chinese open models and the valuation logic of closed labs; engineers who want to judge whether the distillation accusation holds and whether open-source licenses can collect revenue.
The part after 1:04:01 about whether open or closed models are more easily abused — it overlaps heavily with the safety discussion earlier.