True RSI Isn't Here Yet; the Bottleneck Is Peak Intelligence, Not Compute
In-lab automation is concentrated on measurable tasks like software engineering and log monitoring, while scaling laws demand exponential compute for linear intelligence gains—RSI is more likely to make models cheaper than to make intelligence explode.
The video won't play here. Listen to the audio instead:
The argument · timestamps estimated from transcript position
In-lab anxiety is amplified by culture
The author observes that thousands of concurrent agents are already working continuously inside frontier labs, which rapidly inflates employees' expectations about the pace and risk of AI progress. But he argues that the frenzied competitive culture of the San Francisco AI scene itself amplifies any AI concern; fear sells, yet it brings second-order negative effects. He recalls the fierce safety debates in 2023 and 2024 about the existential fate of open-source AI, when the main predicted risks did not arrive on the predicted timeline. Jumping from that anxiety to extinction risk ‘feels very religious’ to him.
Directionally correct but factually wrong timelines
The author cites Richard Ngo's summary: a large portion of the AI safety community is implicitly or explicitly betting on an intelligence explosion within a few years. His own default expectation is that these people will be ‘directionally correct but factually wrong’—no superintelligence within the next 8 years, but things will move fast enough that short-timeline people feel vindicated. He calls this state lossy self-improvement: self-improvement is lossy, not true RSI.
Inference compute expansion is not RSI
From the Dwarkesh and Noam Brown podcast, the author realised that the short-term acceleration from large-scale inference capacity is very substantial, and labs will throw thousands of agents at important and measurable problems. But he doubts that as total compute rises, labs will be able to afford spending a fixed fraction of compute on internal R&D, especially with IPO plans and scrutiny of basic economics. He stresses not conflating the relatively predictable dynamics of inference-time scaling with the highly uncertain output of RSI.
Intelligence is jagged, thresholds are discontinuous
From the three-way discussion between Dwarkesh and John Schulman, Beren Millidge and Charlie O'Neill, the author extracts a key problem: discussions of RSI lack a clear definition of ‘intelligence’. The jaggedness of intelligence means one must discuss thresholds on specific measurable tasks. The shape of LLM intelligence is very different from human intelligence, yet the roles we predict are human-shaped, so AI will not cross thresholds discretely like a remote worker or an AI researcher, but will diffuse slowly, with a long tail forever.
The scientific bottleneck is understanding, not experiments
The author believes many people underestimate the communication and standard-building parts of science. He believes the loop of experiment design and testing will get 10x faster in the near future, but does not believe hypothesis generation and intuition building will. Accelerating understanding is the key bottleneck, and despite major improvements in AI tools, human ability here will only improve marginally. A major advance in science is letting humans spend more time here, not making humans exponentially stronger at it.
RSI makes models cheaper first
The author points out that when it comes to improving AI models, RSI helps efficiency more than scaling peak intelligence. The reason is that LLM serving has clear, measurable, malleable metrics to improve, which brings better inference-time scaling and more efficient multi-agent systems. But all scaling laws show that linear intelligence gains require exponential compute and resources. RSI will make modern LLMs dramatically cheaper, and the trend of exponentially falling cost at a fixed intelligence level may accelerate. For labs the key is improving margins, and the Jevons paradox will likely prevail, bringing a strong business.
Post-training is hard to automate
The author quotes John Schulman: post-training teams need many people because there are many different domains to figure out how the model should behave, and it is hard to automate the whole thing because someone always has to think about how the model should behave in this domain. Schulman also said post-training is easy to screw up in ways that some benchmark cannot see. The author agrees these tasks are especially hard for current LLMs, and although it will get better as RL environments scale rapidly, this paradigm will not last forever, and building and testing new environments that can truly challenge leading LLMs may become exponentially harder in the future.
In-lab automation is concentrated on routine tasks
The author read the internal RSI measurements shared by OpenAI and Anthropic, and his reading is: the biggest takeoff in in-lab automation is concentrated on fairly routine but not always simple tasks like software engineering, monitoring logs, and managing planned experiments. He was especially surprised by the wording in the Claude Fable 5.1 & Mythos 5.1 system card: internal use of recent AI models is a key factor in maintaining the current pace of progress, but no clear signs of acceleration beyond that pace have yet been seen. The author concludes that the hardest exponent to move is peak intelligence, and early RSI is more like massively scaling and diffusing inference-time compute into AI research and related activities, with plenty of low-hanging fruit remaining.
In their own words · checked verbatim
The step from this anxiety, and incidents like OpenAI-HuggingFace, to extinction risks feels very religious.
they’ll turn out to be directionally correct (relative to the expectations of almost anyone not linked to the community) but factually wrong
It is important to not confuse massive steps in inference-time scaling, a dynamic which should be fairly predictable, with being the outputs of RSI, which is highly uncertain.
The jaggedness of intelligence means that we need to discuss thresholds in specific, measurable tasks.
I do buy the cycle of experiment design and testing being 10x faster in the near future, but not hypothesis generation and intuition building.
RSI is poised to make modern LLMs vastly cheaper.
It’s really easy to screw up post-training in some way that doesn’t show up in benchmarks.
John Schulman2:52
We believe that internal usage of recent AI models has been a key factor in maintaining the current rate of progress, but we do not yet see clear signs of dramatic acceleration beyond that rate.
Figures
| Timeline for 10x AI researcher productivity (Charlie O'Neill) | 5–10 years | 2:52 |
| Timeline for 10x AI researcher productivity (John Schulman) | about 2 years | 2:52 |
| Timeline for AI surpassing top human experts at computer work (John Schulman) | 3–4 years | 2:52 |
| Timeline for AI surpassing top human experts at computer work (Charlie O'Neill) | 5–10 years | 2:52 |
| Timeline for AI surpassing top human experts at computer work (Beren Millidge) | about 5 years (lab focus areas) | 2:52 |
| Timeline for remote white-collar job replacement (Charlie O'Neill) | about 1 year (with programmatic tool access); about 2 years (must go through a browser) | 2:52 |
| Timeline for remote white-collar job replacement (Beren Millidge) | about 3 years to full generality; 80–90% coverage earlier | 2:52 |
| Timeline for remote white-collar job replacement (John Schulman) | about 1 year for a ‘decent’ version | 2:52 |
Glossary
- RSI / recursive self-improvement
- An AI system that can autonomously improve its own intelligence, forming an accelerating loop.
- lossy self-improvement
- The author's alternative concept: AI can accelerate parts of the process, but information and understanding are lost in transmission, so a complete RSI cannot form.
- jaggedness of intelligence
- AI capability is uneven across tasks rather than improving smoothly as a whole.
- Jevons paradox
- Efficiency gains lower unit cost, which instead increases total demand and total consumption.
How to listen
Founders, investors and engineers tracking AI progress timelines, especially those who want to calibrate the RSI narrative against the actual degree of automation inside labs.
Readers uninterested in the timeline prediction table from the Dwarkesh podcast can skip the specific year enumeration around 2:52.