AI inference no longer commands 90% margins; each supply layer takes 20-30%
A dollar from customer to Nvidia passes through applications, models, infrastructure, and cloud. Each layer takes 20–30% margin. This is why AI infrastructure companies can't return to software's golden-age profitability.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Capital is not a moat; buying GPU is real cash burn
Modal is profitable on P&L but cash-flow negative—the company burns through cash to secure GPU capacity. This contrasts with Modal's early days, when capital efficiency was extreme (only a few hundred million per month). The shift began a year ago as GPU procurement and capacity-building dominated spending. Erik resists calling capital a moat (the term is overused), but admits it does help at this stage. He also warns that VCs push more capital into companies than they need, causing unnecessary dilution. Early-stage companies should stay lean, take less money, and keep VCs hungry for the next round.
— Erik BernhardssonSelling tokens alone dooms you to a price war
Pure inference has no moat. If you're just converting tokens to dollars, customers have zero switching costs, triggering a race to the bottom. Modal spent five years layering post-training, sandboxes, agent infrastructure, low-latency and batch inference into one complete toolkit—not a single product. The goal: customers buy a full toolbox, not a single-trick pony. The complexity of infrastructure itself becomes the real, hard-to-copy moat.
— Erik BernhardssonThe 90% software margin era is over; stack complexity ensures it
This is an underappreciated structural fact about the inference market. Every dollar flowing from customer to Nvidia traverses a full stack: applications, models, infrastructure, cloud, and chip vendors. Each layer takes 20–30% margin. No company maintains 90% margins anymore. Modal's long-term margin expectation is 40–60%, with no path back to traditional software profitability.
— Erik BernhardssonGPU procurement looks like an open drug market
Erik describes his GTC experience: running between 500 different cloud vendors asking whether they have GPU inventory. He likens the scene to an outdoor drug market, except the product is compute. Modal now works with 25 NeoCloud providers and controls roughly 0.1% of global compute. The long-term play: as Modal controls more GPUs, it can aggregate thousands of customers' fluctuating, unpredictable demand into a smooth curve, then match supply efficiently—a capability Erik believes no one else can replicate.
— Erik BernhardssonSandboxes had a four-year cold start before revenue exploded
Modal launched its sandbox product in 2023. For two years it barely registered. Then summer 2025 arrived—models suddenly got good at code generation and building complete applications—and sandboxes went from zero to $300 million annualized revenue, now roughly one-third of Modal's total. This illustrates an infrastructure pattern: you can't predict which building block becomes a hit. Stable Diffusion and ChatGPT were not foreseen when serverless functions were being built. The only move is to lay good foundational pieces and wait for timing to align.
— Erik BernhardssonDeepSeek made Modal see it had bet on the wrong resource
Modal started late on LLM inference. The team assumed it would be a pure commodity race to the bottom, so it didn't push hard. After DeepSeek's release, the team realized it had to move fast. In one year, it expanded the LLM inference team from one or two people to 30 and is now a top-three provider of open-source LLM inference. When asked whether open-source inference has peaked and frontier models will rebound, Erik disagrees. He rejects the framing—it treats the market as a fixed pie. His view: open-source and frontier inference will each grow 5–20x over the next few years. These are not zero-sum.
— Erik BernhardssonReinforcement learning only works on short-feedback tasks
Here is the boundary of post-training: RL works in coding, network security, and mathematics (with Lean) because they have verifiable, limited feedback loops—code runs in a sandbox and produces a testable result. Whether this scales to longer-feedback cycles is unclear. Software engineers' work has a slower feedback loop, so this part of the job won't be automated soon. That said, Erik has been wrong on predictions like this many times. He expects that work completable within 24 hours or a week will be fully automated, but longer cycles remain safe—for now.
— Erik BernhardssonSelling data to frontier labs is the most profitable business today
Asked what he'd do at age 20 again, Erik says: start a data environment company selling to frontier labs, especially code environments. The money in this space is ‘almost unbelievable’. Caveat: such companies have excellent cash flow today, but whether they support a durable venture-scale business is an open question. His advice: ‘take the money regardless. It's a good business’.
— Erik BernhardssonIn their own words · checked verbatim
You never have moats in the early days.
Erik Bernhardsson10:09
What you used to see in Silicon Valley 10 years ago with this 90% margins plus, I think that's basically gone.
Erik Bernhardsson15:13
I would describe that as like an open air drug market, but GPUs instead of drugs.
Erik Bernhardsson17:16
I actually, I think it's a little overstated. I don't think CUDA itself is the moat.
Erik Bernhardsson23:22
And then Deep Sea came out. We're like, fuck, like we got to move very fast.
Erik Bernhardsson36:36
If I was 20 and I just wanted to make a lot of money, I would go out and start an environment company and just sell data to the Frontier Labs.
Erik Bernhardsson45:49
we've lost new grads going to, you know, some of the labs for like, like talking, you know, 20 plus million.
Erik Bernhardsson52:58
Figures
| Series D funding (Excel-led) | $750 million | 0:00 |
| Modal valuation | $15.75 billion | 0:00 |
| Revenue growth (last 12 months) | 50x | 38:37 |
| Month-over-month growth (last 12 months) | 30%-40% | 20:16 |
| GPU procurement lead time | 3-4 months | 20:16 |
| Sandboxes annualized revenue | $300 million (~1/3 of Modal's total revenue) | 34:35 |
| Claude Code annualized revenue | $60 billion (supporting Anthropic's $2 trillion valuation) | 44:47 |
Glossary
- NeoCloud
- Cloud providers focused on renting GPU compute rather than full-stack cloud services; examples include CoreWeave and Nebius.
- PLG (Product-Led Growth)
- Customer acquisition through self-service product trial rather than sales-team push.
- sandbox
- Isolated execution environment where AI models safely run code and receive immediate feedback on results.
How to listen
Investors and engineering leaders tracking AI infrastructure, GPU supply, and inference market dynamics; founders seeking to understand the true margin structure of infrastructure companies.
Around 4–8 minutes covering childhood and entrepreneurial mindset carries lower information density and can be skipped.