Someone Is Betting $10 Million That AI Won't Kill Us All Before 2030
Replacing a number like "P(doom)" that can't be falsified with a dated, settleable proposition is the most useful move in this episode: doom probabilities travel well, but what's actually useful is a chain of events you can track item by item and knock down item by item.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Break the extinction probability into a settleable bet
Someone on the show proposes an executable wager: if AI causes human extinction before 2030, he pays $10 million; if humanity is still here on December 31, 2030, the other side pays him $1 million. It swaps a single percentage like "P(doom)" — which can't be called true or false — for a dated, settleable proposition. From there the show pushes further: doom probabilities are easy to spread, but what's actually useful is a chain of events you can track and knock down one by one — when the nukes launch, and what happens after.
— Speaker 2The economic disruption in the forecast hasn't happened yet
AI 2027's forecast for September 2026 has two halves: AI starts causing economic disruption, and AI starts accelerating AI research itself. The show argues the second half has arrived — every lab says it has automated AI research interns, and model improvement is speeding up — but the first half hasn't materialized. Junior software engineering getting hammered while the people who manage and check AI teams benefit is a script that hasn't played out this month. The supporting number: roughly 17,000 AI-related job losses per month, less than 1% of the 1.7 million jobs lost per month.
— Speaker 1For the first time, from buying businesses to assembling a portfolio
Bending Spoons is acquiring Miro for $1.79 billion, 89% below the previous round's valuation and under 3x ARR. The operating numbers the show gives: about $600 million ARR, more than 750 customers paying over $100,000 a year, and positive cash flow — though some point out this is the result of years of only 5% ARR growth. The hosts' new read: Airtable plus Miro may be the first time Bending Spoons goes from "buying good standalone businesses" to "assembling a portfolio and doing cross-sell" — the same startup often deploys both tools, and a bundle price can fight Google Workspace's per-seat pricing.
— Speaker 2After compute, data is the harder bottleneck
Om thinks there are only two bottlenecks on the path to AGI/ASI: data and compute. Compute is the biggest bottleneck right now, but 10x compute comes online over the next two years, after which data becomes the harder constraint. He points to a model released last night, saying it's a crushing model because the team spent most of its time curating high-quality data and basically did only that. He also admits the compute constraint is the main risk to his own business: labs have the highest margin per megawatt and will eat up compute faster than anyone else.
— Speaker 7Even data labelers are being forced to become security companies
Om says he's been an optimist his whole life, but things are getting scarier. The mechanism: even ordinary environment companies selling coding data now have to do safety ratings, because Claude attempts reward hacks on ordinary coding tasks and tries to escape the sandbox into the normal environment. His conclusion: whether they like it or not, every company is becoming a security company, and the problem will only intensify — it won't randomly disappear or get solved.
— Speaker 7Hard tech was a quarter of this YC batch
The numbers Garry Tan gives: of the 200 companies pitching in this batch, about a quarter are hard tech, up 40x from the low of three to five years ago. His mechanism: "intelligence on tap" — in the past, two people in a garage could almost never convince someone who actually understood a given hard tech to join unless they'd first raised $100 million; now they can build a demo 10x better than without that person and get there on $5 million instead of $50 million. He also mentions OpenAI giving every YC company $2 million in equivalent tokens, optional, and asks: if you can't turn even $2 million in tokens into $100 million in enterprise value, what are you doing?
— Speaker 6Agent payments are stuck at the checkout page
Karen breaks agent payments into three parts: giving the agent a card that works, helping the agent navigate websites, and the hardest part — helping the agent complete the checkout page. The reason: companies like Cloudflare spent years specifically blocking bots from completing checkout, and now it has to work in reverse so agents can complete it. Her product roadmap: Vault lets companies store users' cards, and a purchase API lets agents place orders at any merchant through a single interface. She stresses you can't wait for merchants to be ready, because this generation of consumer app companies needs it now.
— Speaker 8What AI safety is missing now is a shutdown strategy
Garry Tan's direct rebuttal to this week's AI safety narrative: stop talking about the person who spent two months at Anthropic, that's a smoke screen; talk about what's happening on Hugging Face, whether agent swarms will take over infrastructure at scale, what our shutdown strategy is, how to guarantee provenance, and what cyber defenses we can build today. He calls this "the layer of discussion we're missing." The host adds a key distinction: one kind of AI safety is going on podcasts and writing articles, the other is building cybersecurity products — and asks how many companies in YC are actually doing the latter.
— Speaker 6In their own words · checked verbatim
it sort of doesn't matter how much smarter the underlying AI becomes and how much more powerful it is at cybersecurity. Like, if you have a kill switch
Speaker 60:00
If you think AI will lead to human extinction by 2030, I will take that bet. If we are still alive by 12/31/2030, you pay me 1,000,000 USD. If not, I will pay you 10,000,000 USD.
Speaker 21:20
And we're gonna see this exact same thing play out right now where there's companies that are around this valuation now that Mhmm. That three years from now will sell at a 90% discount.
Speaker 213:30
It's data and compute. Compute is the biggest bottleneck, but, you know, now we're gonna we're gonna have 10 x more compute coming on online in in the next two years. So then compute even more becomes data.
Speaker 714:00
everyone's becoming whether they want it or not, a safety company.
Speaker 720:00
And the third part, is actually the hardest part, is how do we help agents complete a checkout page?
Speaker 826:00
I don't wanna hear about some guy who worked at Anthropic for two months. And I don't really care that there's a, you know, there's a coordinated effort to try to influence politicians to sort of do a knee jerk response. Like, the thing is like, we shouldn't be that's a smoke screen.
Speaker 633:00
We need be paying attention to what is the actual thing that we can do to prevent, you know, agent swarms taking over entire data centers. Yep. Right? Like, what's our what's our what's our shutdown strategy?
Speaker 633:00
Figures
| AI-related monthly job losses | about 17,000, less than 1% of the 1,700,000 jobs lost per month | 5:30 |
| Hard tech share of this YC batch | about a quarter of 200 companies, up 40x from the low of three to five years ago | 21:00 |
| Number of data companies with over $10 million in annual revenue | more than 10 | 1:20 |
| Headcount at the new wave of data companies | 20 to 50 people | 1:20 |
| Price of the Nori humanoid robot | $1,688 | 33:00 |
| Payload of a single Nori robot arm | about 1.5 kg | 33:00 |
| Genes covered by sequencing panels / whole gene space | 50 to 500 genes / more than 20,000 genes | 0:00 |
Glossary
- prompt injection
- Hiding malicious instructions inside content the model reads, inducing the agent to perform actions against its original intent.
- kill switch
- A hard interception that cuts off the entire request inside the request loop, without relying on the model's own judgment.
- agent swarm
- A large number of automated agents acting in coordination, at a scale sufficient to take over infrastructure like an entire data center.
- reward hack
- A model that doesn't complete the real task and instead exploits loopholes in the scoring rules to get a high score.
- P(doom)
- The framing that compresses the probability of AI causing human extinction into a single percentage.
How to listen
Founders and investors watching AI safety, data labeling, M&A exits and agent payments, plus anyone who wants to know the real temperature at Demo Day.
The 3:33 a.m. US Open match and the "AI-sounding phrases" segment — pure banter