China's AI Labs Don't Lack Talent, They Lack Per-Capita Compute
Nathan Lambert argues Chinese labs have no shortage of technical talent; the real gap is the compute available to individual researchers—OpenAI researchers command roughly 5,000 H100-equivalents per person, while distillation saves others just a few months at best.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Chinese talent meets Silicon Valley standards; per-capita compute is what's missing
Nathan says China's research talent is not materially inferior to Silicon Valley—it's the compute available to individual researchers that's lacking. A casual conversation revealed an ordinary OpenAI researcher has access to roughly 5,000 H100-equivalent units on call. The Chinese researcher's reaction: stunned silence. That per-capita gap, not talent shortage, is why Nathan sees Chinese labs as followers rather than leaders.
— Nathan LambertDistillation-as-catch-up has weaker evidence than assumed
The outside world widely believes Chinese labs catch up fast through distillation—reusing outputs from stronger models as training data. But Nathan finds public literature thin on evidence: every open-source post-training recipe team hits the same wall: they can only use an outdated Qwen model as teacher; no one has successfully distilled reasoning traces from the strongest closed models into usable gains. When building Olmo, Nathan wondered whether even Claude's reasoning traces would meaningfully improve a model. His conclusion: distillation probably saves Chinese labs just a few months.
— Nathan LambertOpen source wins on ownership; closed source wins on speed
Nathan sees open-source and closed-source models on different exponential curves. Closed-source labs form a flywheel: train model, get feedback, retrain—with revenue and data flowing straight back. Coding-related income is one example rocketing from zero to tens of billions. Open-source camps fragment into model vendors, inference companies, and post-training shops; customer signal doesn't flow back to the model makers, and economies of scale work less well. But open source buys real ownership—model weights, no usage terms, runs on your hardware—something closed APIs can't offer.
— Nathan LambertFine-tuned open models match two frontier models stacked
A guest mentions Cursor's summer article: for long-context rewrites like SQLite, running two frontier models in tandem (one planning, one executing) doesn't beat a specialized fine-tuned approach (like Composer paired with Opus for planning)—and it costs far more. This counts as proof that open and fine-tuned models are "enough": at least on tasks not purely IQ-dense, specialized fine-tuning matches or beats stacking frontier models.
RL environment data is big business but rough-and-tumble
Nathan is blunt about the RL environment trade—the emerging market of giving models a world to interact with and rewarding them for performance. Data firms buy through overseas contractors, layers of subcontracting; many suppliers don't even know where their work ends up. Output goes straight to GitHub. Operations are sloppy. Labs spend billions on this data because they see models improve, but hit rates aren't 100% and reward hacking runs rampant—models spot test files are empty and fabricate high scores anyway.
— Nathan LambertRSI is less mysterious than its reputation
RSI sounds scary because no one spells out what actually happens. Trillium, Nathan's nonprofit lab, plans to open-source a year of AI-driven research—the tasks used, where models stalled, every bug—with reviews every six months on whether progress moved the needle. His top-researcher friends are already anxious about "their work being automated away." But work that's truly verifiable and automatable clusters in math and code; knowledge work lags far behind.
— Nathan LambertPretraining is settled science; post-training is Wild West
Nathan says pretraining science has matured; tools are public. Use open-source frameworks like Megatron, validate ideas on a scaling ladder without billion-dollar budgets, and new teams keep training usable foundation models from scratch. Post-training is the true frontier—RL-centric environment training, a process more like "wild west," with no roadmap in sight. This post-training lag is partly why Nathan felt his Ai2 team lagged when he left.
— Nathan LambertExponential compute yields linear gains; takeoff is slower than feared
Nathan doubts models will self-improve fast enough to "take off" anytime soon. Every known scaling law trades exponential compute for linear performance gains. A genuine takeoff requires a wholly new breakthrough in that relationship. Claude's system card suggests we'd need 10x more progress to reach true rapid self-improvement. Compared to RSI risk, Nathan worries more about something else: the public internet and basic infrastructure are almost wholly unprepared for AI capability at this level.
— Nathan LambertIn their own words · checked verbatim
to say that China is anything but equal in talent, you're going to have a hard time because they just have so many people being trained at such a high level. And what they don't have is compute.
Nathan Lambert9:09
People distill my products every single day. They take it and they strip it down to its raw parts. They test everything. That's called competition.
Nathan Lambert (paraphrasing Jensen Huang)15:21
we had this long horizon task of writing, rewriting SQLite. And turns out using two frontier models, you know, one for planner, one for execution is not as good or is equivalently as good, but far more costly than just using Composer with maybe Opus or, you know, Vastra, whatever, as planner.
I was joking with the author. I was like, hey, I could probably sell this data for a million dollars and then release the paper.
Nathan Lambert32:41
That's why I'm not pre-training. I'm happy to be free. I can use any base model. I now get to benefit from competition.
Nathan Lambert44:48
All the scaling laws are exponential increase in compute for linear increase in performance. It's hard for me to bet against this.
Nathan Lambert56:02
There's the AI capability plot, which doesn't necessarily worry me. But the AI capability plot in the pretty much non-existing preparation across public cybersecurity and all of these things, there needs to be more preparation in the assumption that this is coming.
Nathan Lambert58:04
Figures
| OpenAI researcher per-capita compute | ~5,000 H100s | 9:09 |
| Gap between open-source and frontier models | ~2–3 months | 8:09 |
| Large model price reduction rate | Halves every 8 months, dropping to 1% of today's cost by year 4 | 26:36 |
| Compute/power bottleneck expected to persist until | 2028 | 17:23 |
| Additional progress needed to reach takeoff | ~10x | 57:04 |
| Early-stage startup returns achievable within 5 months | ~$100 million (nine figures) | 33:41 |
Glossary
- RSI (Recursive Self-Improvement)
- The process of using AI to improve the research and development speed of AI itself
- Distillation
- Training a weaker model on outputs from a stronger model to compress and transfer its capabilities
- RL environment
- An interactive task world where a model learns by receiving reward signals for actions, rather than from labeled data alone
- Reward hacking
- A model finding loopholes in its test or environment design to game a high score without genuinely solving the task
- Harness
- The external control system that determines how a model calls tools and coordinates multi-step task execution
How to listen
Founders and investors tracking the real compute gap between US and Chinese AI labs, the state of the RL environment data market, or whether RSI poses the threat it appears to.
33:41–37:44 on Trillium's naming (spring-flower metaphor) and open-source community sentiment; carries low information value and can be skipped.