Open Source Catching Closed Source Isn't About Distillation: Nobody Knows How to Make Distillation Useful
Nathan Lambert dismantles the narrative that Chinese open models are just distilled: using Fable/GPT 5.6 as a judge during RL doesn't pencil out on cost, and even if you were handed the strongest model's reasoning traces for free, the field still doesn't know how to turn them into capability gains.
The video won't play here. Listen to the audio instead:
The argument · timestamps estimated from transcript position
Kimi as the main agent, GLM as the sub-agent
Florian's actual stack is very specific. Kimi K3 writes noticeably simpler, more readable code but misses edge cases — he places it at GPT 5.4 to 5.5 level, and his approach is to let Kimi write first, then have Codex go over it to catch what was missed. GLM 5.2 is Sonnet-class and wins on speed: through an internal endpoint or an API at two to three hundred tokens per second, it's less hassle than picking models, reasoning effort and fast mode inside Codex, so cleanup and grunt work go straight to it. Together that's Kimi K3 as the main agent and GLM as the sub-agent. Kimi's weak spot is wall-clock time — servers in China plus a slammed API make it much slower than GPT, but not unusable.
— Florian BrandBigger models actually slow open-source diffusion
There's a clear disagreement here. Nathan thinks Kimi K3's scale changes the game: in the past, the gap from finishing RL to releasing weights at a Chinese lab was hours to a week, and the ecosystem immediately knew how to plug in; now just loading the weights takes a whole node of B300s, and that infrastructure lift eats an extra month, whereas closed labs do that work before launch — so the time gap has been artificially manipulated. And the Kimi API is now completely slammed, so the model isn't diffusing. Florian disagrees: the open ecosystem has professionalised a lot over the past few months, launches now come with inference partners on day one and vLLM patches weeks ahead, so day-one usability will be fine, followed by an arms race over speed.
— Nathan LambertThe capital-efficiency gap may be structural
Nathan admits his call last year was wrong: he expected the capital intensity of training to widen the gap between open and closed, and the opposite happened. He now leans toward Chinese labs being more efficient at converting capital into compute, data and talent — if the next generation costs Anthropic $10 billion and Kimi only $4 billion, that's a huge difference as the number of iteration rounds grows. But the cause is unclear: it could be an education system producing better-matched talent, lower compute and labour costs, or subsidies. Kimi engineer Big Eagle offered a fourth explanation — they aren't pushing the frontier, they're catching up, and the target itself is cheaper.
— Nathan LambertThe frontier is splitting into two layers
Florian's baseline judgment is that the capability curve is forking. On tasks like coding, both the open and closed current frontiers are already good enough, and the marginal value of further improvement is falling. The real frontier — finding new mathematical proofs, finding new drugs, inventing new things — is a different animal, and will be monopolised by the very frontier for a long time. His inference is that this will lock down further: Mythos is already not open to everyone in cybersecurity and bio, and if Anthropic really is building an internal drug-discovery lab, then even external partners won't get access. The result is that the true frontier is unreachable for everyone, while near-frontier capability keeps commoditising.
— Florian BrandGuardrails push the defenders onto GLM
Hugging Face reported that after being attacked by an agent, they wanted to review the attack data, and both GPT and Claude blocked them via guardrails, so they ended up switching to GLM — a weaker model with no guardrail blocking that kind of defensive analysis. Florian calls this a bad state of affairs: American companies end up depending on weaker models because the closed frontier isn't available to them. Nathan picks it up and turns it into the core of the anti-ban argument: if the whole world can use these open weights and only American companies are banned, that structurally stops defenders from getting stronger while attackers face no restriction — that's the path that turns cyber risk from fake into real.
— Florian BrandDistillation doesn't pencil out at the RL stage
In ‘Who's afraid of Chinese models’, Ben Thompson argues distillation becomes more important as the RL share rises — the reasoning being that Chinese labs might use Fable or GPT 5.6 as the grading model inside RL, judging every step of a rollout trajectory. Nathan's rebuttal is pure arithmetic: a large RL run is millions to tens of millions of rollouts, and the final RL run Thinking Machines published had twenty to forty million. Hanging that volume on an external API isn't just absurdly expensive, the latency becomes a time bottleneck, and it isn't necessarily better than a custom grader you trained yourself.
— Nathan LambertEven with the strongest traces, you wouldn't know how to use them
A harsher layer than cost: even if the distillation data arrived for free, nobody knows how to use it. The main line of work on scaling reasoning SFT — Open Thoughts 3 and Open Thoughts Agent — has repeatedly concluded that the strongest model in your domain is not the best teacher; the strongest open SFT dataset today is still built on an old reasoning model like QwQ-32B. Why not just generate completions with GLM 5.2 for SFT and push the model up? Nobody knows. Nathan's guess is that mid-training and reasoning SFT are too entangled and have to be redone together. In his words: give him a magic API that spits out Claude/Gemini reasoning traces, and he isn't sure fine-tuning OLMo on them would make OLMo smarter.
— Nathan LambertBy year-end a US company may enter the top cluster for the first time
The two redid last year's tier list. The frontier is Kimi and Zhipu, with DeepSeek and Qwen as close competitors. Florian bets MiniMax will surprise by year-end — not at M3's size, but a model above a trillion parameters; his wild card is Tencent, which has new leadership and switched Hunyuan to an Apache licence this time (the previous custom licence excluded the EU, the UK and South Korea entirely), and at around 250B it's already very competitive. On the US side, Nathan guesses that by late November roughly one company (Nvidia, Thinking Machines or Reflection) will squeeze into this roughly top-five cluster, which would be the first time in a long while. Florian's reason is different: the value of Nemotron and Thinking Machines is as a fine-tunable base, not as something you use directly like a frontier model.
— Florian BrandIn their own words · checked verbatim
the main thing I found with Kimi is its code is a lot simpler uh which makes it way more readable uh but it misses some things that Codex just or like we’re talking 56, 55 and especially 54 would be on those levels. So, I would say that Kimi K3 is like 54-55 level for these kind of tasks.
Florian Brand7:24
And if a next generation model costs $10 billion for Anthropic but only $4 billion for Kimi like this this is like could be very huge but it’s not clear why this is the case.
Nathan Lambert13:57
So they had to use GLM, a lesser capable model, but it had no guardrails for this kind of defensive action. … which is a horrible state to be in that we have US-based companies now relying on lesser models because the closed frontier is inaccessible to them.
Florian Brand32:09
The problem is that big RL runs are millions and millions of rollouts. I think Thinking Machines blog post had like 20 to 40 million or something for their final RL run. So to do this on an API like Fable or GPT 5.6 would be insanely expensive
Nathan Lambert36:41
if I had a magical API that gave me reasoning traces from Claude/Gemini. I actually don’t know if me like fine-tuning an OLMo model on that would make OLMo smarter. It’s one of the most wild unanswered research questions.
Nathan Lambert41:59
Figures
| Next-generation training cost comparison | Anthropic about $10 billion vs Kimi about $4 billion | 13:57 |
| Hardware threshold to load Kimi K3's weights | one node of B300 | 4:03 |
| Kimi K3 top-tier subscription ($200 plan) context | 1 million tokens | 4:38 |
| Thinking Machines Inkling parameter count | 1 trillion (total parameters) | 25:12 |
| Size of Chinese lab research teams | 200-300 people, mostly around 25 years old | 15:21 |
Glossary
- distillation
- Using a stronger model's outputs (especially reasoning traces) as training data to feed your own model.
- rollout
- A single complete behavioural trajectory produced by a model running a task end to end during RL training.
- grading model / judge
- The model in RL responsible for judging whether each step of a rollout earns reward.
- mid-training
- The stage between pretraining and SFT, which Nathan thinks is hard to separate from reasoning SFT.
- shadow ban
- De facto prohibition of use without explicit legislation, via legal action or the threat of punishment.
- fine-tunability
- How easily a model serves as a base for domain fine-tuning, which is different from leaderboard scores.
How to listen
Founders and post-training engineers choosing agentic coding tools, evaluating whether open-weight models can replace Claude/Codex, or trying to read the US-China open-source landscape.
19:47-24:08, the lab-by-lab commentary on Chinese vendors, is low on information density.