Math advanced in two years from failing arithmetic to solving millennium problems; AI research accelerates at the same pace
The author judges that AI's progress on verifiable math problems over two years implies AI research itself can be tackled through RL. Median forecast: 2028; fastest possibility: next year. Superhuman AI research agents will emerge.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Anthropic's research pace has doubled, according to independent measurement
Independent auditor Meter spent three months embedded in Anthropic and concluded that the company's AI research progress on clearly-defined tasks has doubled compared to a year ago, with this acceleration ‘very likely mainly attributed to’ heavy use of Claude. Anthropic disclosed that Claude now drives 26% of the company's AI research work, up from less than 1% in February. The author treats this as direct evidence that AI is already substantially accelerating AI research, not a future prospect.
Models advanced from arithmetic errors to solving millennium problems in two years
The author lays out a timeline: in 2023 models failed reliably on four-digit addition; in 2024 they could barely solve high school math competition problems; in 2025 they won a gold medal at the IMO; a month ago OpenAI used 88 hours to solve a millennium prize problem. He quotes East of Eden—‘when the gods fall, they do not fall slowly, they shatter like glass’—to describe this cliff-edge acceleration. Even if most progress concentrated on ‘finding counterexamples’ and other verifiable work rather than proposing new theory, a system that can solve millennium problems is remarkable in itself.
Transformers could have been discovered two years sooner
The author relays an observation from friend Sholto Douglas: before 2017, the core Transformer components—attention mechanisms, residual networks—had each been validated by researchers, and the active researcher Chris Olah considered this a ‘natural combination of known ideas.’ The real bottleneck was not inventing new concepts but finding the right way to combine half a dozen known techniques. This combination-finding is precisely the strength AI demonstrates in mathematics, and why the author believes models like Claude could have discovered Transformers earlier.
Superhuman AI researchers will likely arrive by 2028
The author's median forecast is 2028—assuming labs don't deliberately slow down and governments don't intervene—when automated research agents reach the level of top human AI researchers. He sees at least a 25% chance this slips to 2032 or later, but also flags a 2027 possibility: if you directly apply Meter's ‘AI research pace doubled’ finding to a takeoff model, the math points to reaching extreme superintelligence within a year. He remains skeptical of simple extrapolation accuracy, however.
Pure software singularity and Spark scenario are roughly equally likely
The author distinguishes two scenarios: if AI becomes smarter faster than ‘good ideas become scarce,’ a pure software singularity emerges—sustained self-improvement without added compute. Otherwise comes what he calls ‘Spark’—three to six years of progress compressed into a calendar year, followed by gradual slowdown as experiments demand more compute. He assesses these roughly 50-50 probability, but even the conservative Spark case means compressing GPT-3.5-to-GPT-6-scale advancement into a single year.
Every stage of AI development has clean verifiable metrics
The author divides AI development into three parts, arguing each has verifiable proxy metrics: pretraining via direct RL to minimize loss on held-out test sets; post-training by measuring how much data, time, or compute a model needs to learn a new environment, or how quickly it masters novel tasks; inference efficiency through GPU utilization, KV cache size, and flops per forward pass. He notes AI lab Magi recently pre-trained a DeepSeek v4-competitive model with approximately $500,000 of compute, evidence that pretraining remains far from saturation.
Continual learning was the main barrier to AI research automation—now it's not
The author admits that 18 months ago he saw continual learning—where models accumulate experience on the job rather than resetting between conversations—as the biggest obstacle to automating AI research. His current view: expanding context windows may suffice to simulate equivalent long-term learning, and continual learning itself is an ML research problem with clear verifiable metrics, no harder than millennium problems. Existing models solved millennium problems without persistent memory anyway.
Compute abundance means ten thousand researcher agents can operate simultaneously
Against the objection that more autonomous researchers means less experimental compute per researcher, the author estimates that if frontier labs keep doubling compute annually, by end-2027 OpenAI could let approximately 10,000 agents each run a GPT-3-scale training daily—the scale that solved millennium problems. He notes current researcher habits reflect ‘human labor scarcity, not absolute compute limits,’ so compute won't become a hard bottleneck for research automation.
In their own words · checked verbatim
And there is one sure thing about the fall of gods. They do not fall a little. They crash and shatter.
Dwarkesh (quoting from novel)3:09
My sense is that around 80% of the algorithmic progress at a given frontier lab is driven by its 50 top researchers.
Dwarkesh12:16
GPT-3.5 was able to ride a passable limerick and autocomplete a trivial Python function, but otherwise it was mostly a useless toy.
Dwarkesh16:20
By the age of 27, Napoleon had stacked victory after victory against far larger armies from the other great powers of Europe.
Dwarkesh23:37
These models are out here solving Millennium Prize problems while still being these derpy savant interns that have no long-term memory. It's like watching somebody do cartwheels down the track and still keep pace with Usain Bolt.
Dwarkesh35:58
the gods above must work from the same principles as the ants below.
Figures
| Anthropic's AI research pace (Meter measurement) | 2x compared to one year ago | 1:01 |
| Claude's share of Anthropic AI research work | 26% (February: less than 1%) | 1:01 |
| Time for AI to solve a millennium prize problem | 88 hours | 3:09 |
| Median year for superhuman AI researchers | 2028 | 8:10 |
| Probability the milestone delays past 2032 | at least 25% | 9:11 |
| Data improvement vs. model improvement (six years) | more than 3x | 39:04 |
| Global AI compute annual growth rate (past five years) | approximately 3x | 24:38 |
| Research agents OpenAI can support by end-2027 | approximately 10,000 (each running GPT-3-scale training daily) | 40:07 |
Glossary
- RSI
- AI automatically participates in improving the next generation of AI training and development, accelerating its own capability gains.
- Spark scenario
- Software advances compress three to six years of progress into a single calendar year, then slow as compute constraints bind.
- software-only singularity
- Sustained self-improvement through algorithmic and software advances alone, without increases in computational resources.
- continual learning
- An AI model accumulates new capabilities from deployment experience over time, rather than resetting between conversations.
- sample efficiency
- How much more training data an AI system requires compared to humans to acquire the same capability.
- MuP
- A scaling method that predicts optimal hyperparameters for large models based on small-scale model experiments.
How to listen
AI industry practitioners, investors and researchers tracking R&D automation progress and recursive self-improvement reasoning. Anyone curious about whether AI research is verifiable and tractable, like mathematics.
If you're not interested in technical details of verifiable proxy metrics, you can skip 27:48–33:00, which contains an appendix-style argument about pretraining, post-training, and inference efficiency.