Can a Small Model Grinding for 8 Weeks Break the Foundation Model Oligopoly?
Poolside used a small model with 118B total parameters and 8B active to show that persistence and verification behavior improve coding ability more than simply stacking parameters, leading them to believe open small models can break the giants' monopoly.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
One blog post, $12 million burned four years later
In 2015, Karpathy's blog post on RNNs made him pivot his company overnight to using neural networks to write code. Over the next four or five years, the team tried RNNs, LSTMs, and Transformers. The direction was right, but they didn't realize the only thing they needed to do was keep scaling. The team burned $12 million of investor money, which he calls the biggest failure of his career. For the next two years he barely touched language models, until ChatGPT launched and he got some kind of belated vindication.
— Eiso KantRather 100 foundation model companies than 5
When Poolside was founded, like most labs, it didn't plan to go open source. The turning point came early this year: they realized the world was heading toward a script where only three or four companies decide all intelligence futures, which to him felt more like a dystopian novel than a sci-fi ideal. Because they weren't yet at the frontier, the cost of this decision wasn't too high—he admits that if they were at the frontier, they probably couldn't turn back. So they chose to open research and weights, for a direct reason: they'd rather live in a world with 100 foundation model companies than only 5, even if they were one of those 5.
— Eiso KantModel factory: train a model in 8 weeks
Poolside treats model training as an industrial process: three years ago, the industry standard was to package datasets, copy them to training clusters, then distribute; any data change required repackaging, which was extremely time-consuming. They switched to streaming data into training, keeping the data layer immutable, with every token traceable to a specific code version, so experiments from two years ago can still be fully reproduced. The team has fewer than 70 researchers and 35 engineers, running 10,000 to 20,000 experiments per month: Laguna XS2 went from training start to release in 5 weeks, Laguna S in 8 weeks, and as soon as S was done, they started training the larger Medium the same day. What he sees now is mostly agents writing code, running experiments, evaluating results—early signs of recursive self-improvement.
— Eiso KantSmall models grind, not get smarter
Peng Ming, head of applied research, told Eiso that Laguna S's progress came more from behavior changes than stronger intelligence: more verification, not prematurely claiming success, and grinding on problems more persistently. This small model with 118B total parameters, 8B active parameters, and able to fit on a single DGX Spark independently solved the Erdős 397 problem, and without internet, by repeatedly probing low-level APIs, wrote a Wi-Fi scanner. This made Eiso start to suspect that the cost-performance inflection point for model scale needed for knowledge work (about 25% of the global economy, $25 trillion) might be far lower than he previously imagined—but he stresses this is not an argument against continuing to build bigger models, just that there's still a lot of untapped space in small models.
— Eiso KantDoubts that stacking RL environments is the wrong path
Eiso believes reinforcement learning will intervene earlier and earlier in the training process, not just as post-processing after training—he sees DeepSeek's R1-Zero paper as the industry's earliest public signal that reasoning behavior can be induced as soon as a model learns to use language. He bluntly says the industry currently has two addictive "drugs": one is distillation, the other is constantly piling on more RL environments. Both make models better and make people feel good, but he admits he's not sure this is the right path to AGI, nor whether the information in web data has already been fully extracted by next-token prediction.
— Eiso KantMCP and tool calling are "dumb" designs
Eiso has held an unpopular view for two years: MCP and traditional tool calling are actually dumb. His logic is that complex long-horizon tasks are essentially about dealing with data sources and systems. Rather than inserting an MCP or tool-calling layer between the model and the system, it's better to give the model a code repository with all kinds of libraries, documentation, and API keys installed, and let it write code itself, using if and for loops to complete tasks—RL-trained models instinctively want to do this. He predicts that within 12 months, no one will stuff 20 or 30 tools into system prompts, but Poolside currently still supports tool calling and parallel tool calling because the industry hasn't generally caught on yet.
— Eiso KantRestricting open models may create a new oligopoly
Eiso opposes any single company unilaterally deciding who can use its models to develop foundation models—he believes such restrictions should be left to democratic decision-making, not a few companies. He gives the example that banning cigarette advertising is right in itself, but objectively created an oligopoly in the tobacco industry, because new entrants could no longer build brands through advertising. He worries a similar lock-in effect will happen in AI: once restrictions on open models begin at the current level of model capability, it's like passing a death sentence on the future in advance, and he thinks current model capabilities are far from dangerous enough to warrant that.
— Eiso KantRL bottleneck isn't GPU count, it's batch size
Pretraining can use the entire internet as data, and batch size can be almost infinitely expanded; but RL training is limited by the finite number of tasks/environments, so batch size can't be scaled that way, making RL training time the real bottleneck, not GPU count itself. Eiso is therefore eager to bring the idea of separating prefill/decode in inference and using specialized hardware for different stages into RL training, and mentions a blog he saw a few days ago about doing RL training with lower precision than Poolside's current FP8. He repeatedly stresses: this is a race measured in calendar time, not GPU count, and shortening the wall-clock time of RL training may be the biggest lever to speed up the entire industry right now.
— Eiso KantIn their own words · checked verbatim
I had built a completely unreasonable belief, that neural nets should be able to generalize to anything and everything, and that language should be able to generalize, to a lot of things that are intelligent and the ability to write code.
Eiso Kant1:05
I rather live in a world that has 100 foundation model companies than a world that has five, even if I was one of the five.
Eiso Kant6:21
The model that we’re gonna talk about today was eight weeks from start of training, to launch. We started the next model literally yesterday because we now finished the post-training required for the model we’re launching, next week or by the time this comes out today.
Eiso Kant18:18
I have the feeling that a lot of the gains in Laguna S come not from more intelligence, but more from different behavior, more verification, less taking things for granted, not declaring victory early, and being way more persistent.
Eiso Kant35:46
we’ve got some drugs in the industry. One of the drugs is distillation. Another drug is, more environments. Like, and they’re great, and they make us feel good, and they make the models better, and like we’re all addicted to them, and we’ll use them, right?
Eiso Kant44:39
I think MCP and tools are stupid.
Eiso Kant1:09:28
I always think back about when we banned advertising on cigarettes. Good thing. I’m not saying I’m against that. But it effectively established an oligopoly of cigarette companies because no one else could ever compete.
Eiso Kant1:31:53
The race is measured not in how many GPUs, but the race is measured on calendar time, and that’s probably one of the biggest impacts we can have right now to speed up our industry.
Eiso Kant1:36:25
Figures
| Sourced failure burned investment | $12 million | 2:29 |
| Poolside researchers/engineers | Fewer than 70 researchers, plus 35 engineers | 18:18 |
| Laguna XS2 training cycle | 5 weeks from training start to release | 18:18 |
| Laguna S training cycle | 8 weeks from training start to release | 18:18 |
| Laguna S parameter scale | 118B total parameters, 8B active parameters | 54:34 |
| Knowledge work share of global economy | About 25%, or $25 trillion | 38:05 |
| New generation Laguna Medium training cycle | 39 days | 1:18:19 |
| Poolside funding | $500 million | 1:23:09 |
Glossary
- RSI / Recursive self-improvement
- The ability of AI systems to improve the AI R&D process itself, seen as a signal of approaching AGI
- MCP / Model Context Protocol
- A standard protocol for models to call external tools/data sources; Eiso thinks it's better to let models write code directly
- FP8/NVFP4 / 8-bit/4-bit low-precision training
- Training models with lower numerical precision to save compute, one of the directions Poolside is betting on
- DGX Spark / Nvidia desktop AI compute device
- A small compute box that can run tens-of-billions-parameter models on a single device
- prefill/decode disaggregation
- Splitting prompt processing and response generation onto different hardware in inference; Eiso wants to bring this into RL training
How to listen
Research/engineering leads at foundation model companies, and investors watching the open-source model ecosystem and RL training bottlenecks.
Around 1:20:40, the origin of the Poolside name and fundraising anecdotes are low-information and can be skipped.