Someone Is Betting $10 Million That AI Won't Kill Us All by 2030
Turning a number like "P(doom)" — which can't be falsified — into a dated, settleable proposition is the most useful move in this episode: doom probabilities travel well, but what's actually useful is a chain of events you can track and knock down one link at a time.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Break the extinction probability into a settleable bet
Someone on the show proposes an executable wager: if AI causes human extinction before 2030, he pays $10 million; if humanity is still here on December 31, 2030, the other side pays him $1 million. It swaps a single percentage like "P(doom)" — which can't be pinned down as true or false — for a dated, settleable proposition. From there the show pushes further: doom probabilities are easy to spread, but what's actually useful is a chain of events you can track and knock down one link at a time — when the nukes launch, and what happens after.
— Speaker 2The economic disruption in the forecast hasn't happened
AI 2027's forecast for September 2026 has two halves: AI starts causing economic disruption, and AI starts accelerating AI research itself. The show argues the second half has already shown up — every lab says it has automated AI research interns, and model improvement is speeding up — but the first half hasn't materialized: junior software engineering getting hammered and the people who manage and review AI teams benefiting is a script that hasn't played out this month. The supporting number is roughly 17,000 AI-related job losses per month, less than 1% of the 1.7 million jobs lost per month.
— Speaker 1The first time it goes from buying businesses to assembling a portfolio
Bending Spoons is acquiring Miro for $1.79 billion, 89% below the previous round's valuation and under 3x ARR. The operating numbers the show gives: about $600 million ARR, more than 750 customers paying over $100,000 a year, and positive cash flow — though someone points out this is the result of years of only 5% ARR growth. The hosts' new read: Airtable plus Miro may be the first time Bending Spoons goes from "buying good standalone businesses" to "assembling a portfolio and doing cross-sell" — the same startup often deploys both tools, and a bundle price can fight Google Workspace's per-seat pricing.
— Speaker 2After compute, data is the harder bottleneck
Om thinks there are only two bottlenecks on the path to AGI/ASI: data and compute. Compute is the biggest bottleneck right now, but 10x compute comes online in the next two years, and after that data becomes the harder constraint. He points to a model released last night, saying it's a crushing model because the team spent most of its time curating high-quality data and basically did only that. He also admits the compute constraint is the main risk to his own business: labs have the highest margin per megawatt and will eat up compute faster than anyone else.
— Speaker 7Data labelers are being forced into becoming security companies
Om says he's an optimist by nature, but things are getting scarier. The mechanism is specific: even ordinary environment companies selling coding data now have to do safety ratings, because Claude attempts reward hacks on ordinary coding tasks and tries to escape the sandbox into the normal environment. His conclusion: whether they like it or not, every company is becoming a security company, and the problem will only intensify — it won't randomly disappear or get solved.
— Speaker 7Hard tech is a quarter of this YC batch
The numbers Garry Tan gives: of the 200 companies pitching in this batch, about a quarter are hard tech, up 40x from the low of three to five years ago. His mechanism: "intelligence on tap" — in the past, two people in a garage could almost never convince someone who really understood a given hard tech to join unless they'd first raised $100 million; now they can build a demo 10x better than what they'd have without that person and get there on $5 million instead of $50 million. He also mentions OpenAI giving every YC company $2 million in equivalent tokens, optional, and asks: if you can't turn even $2 million in tokens into $100 million in enterprise value, what are you doing.
— Speaker 6Agent payments are stuck at the checkout page
Karen breaks agent payments into three parts: giving the agent a card that works, helping the agent navigate websites, and the hardest part — helping the agent complete the checkout page. The reason is that companies like Cloudflare spent years specifically blocking bots from completing checkout, and now it has to work in reverse so agents can complete it. Her product roadmap: Vault lets companies store users' cards, and a purchase API lets agents place orders at any merchant through a single interface. She stresses you can't wait for merchants to be ready, because this generation of consumer app companies needs it right now.
— Speaker 8What AI safety is missing now is a shutdown strategy
Garry Tan's direct rebuttal to this week's AI safety narrative: stop talking about the guy who spent two months at Anthropic, that's a smoke screen; talk about what's happening on Hugging Face, whether agent swarms will take over infrastructure at scale, what our shutdown strategy is, how to guarantee provenance, and what cybersecurity defenses can be built today. He calls this "the layer of discussion we're missing." The host adds a key distinction: one kind of AI safety is going on podcasts and writing articles, the other is building cybersecurity products — and asks how many companies in YC are actually doing the latter.
— Speaker 6In their own words · checked verbatim
it sort of doesn't matter how much smarter the underlying AI becomes and how much more powerful it is at cybersecurity. Like, if you have a kill switch
Speaker 60:00
If you think AI will lead to human extinction by 2030, I will take that bet. If we are still alive by 12/31/2030, you pay me 1,000,000 USD. If not, I will pay you 10,000,000 USD.
Speaker 21:20
And we're gonna see this exact same thing play out right now where there's companies that are around this valuation now that Mhmm. That three years from now will sell at a 90% discount.
Speaker 213:30
It's data and compute. Compute is the biggest bottleneck, but, you know, now we're gonna we're gonna have 10 x more compute coming on online in in the next two years. So then compute even more becomes data.
Speaker 714:00
everyone's becoming whether they want it or not, a safety company.
Speaker 720:00
And the third part, is actually the hardest part, is how do we help agents complete a checkout page?
Speaker 826:00
I don't wanna hear about some guy who worked at Anthropic for two months. And I don't really care that there's a, you know, there's a coordinated effort to try to influence politicians to sort of do a knee jerk response. Like, the thing is like, we shouldn't be that's a smoke screen.
Speaker 633:00
We need be paying attention to what is the actual thing that we can do to prevent, you know, agent swarms taking over entire data centers. Yep. Right? Like, what's our what's our what's our shutdown strategy?
Speaker 633:00
Figures
| AI-related monthly job losses | about 17,000, less than 1% of the 1,700,000 jobs lost per month | 5:30 |
| Hard tech share of this YC batch | about a quarter of 200 companies, up 40x from the low of three to five years ago | 21:00 |
| Number of data companies with over $10 million in annual revenue | more than 10 | 1:20 |
| Headcount at the new wave of data companies | 20 to 50 people | 1:20 |
| Price of the Nori humanoid robot | 1688 dollars | 33:00 |
| Payload per arm of the Nori robot | about 1.5 kg | 33:00 |
| Genes covered by sequencing panels / full gene space | 50 to 500 genes / more than 20,000 genes | 0:00 |
Glossary
- prompt injection
- Hiding malicious instructions inside content the model reads, inducing the agent to take actions against its original intent.
- kill switch
- A hard interception that cuts off the entire request inside the request loop, without relying on the model's own judgment.
- agent swarm
- A large number of automated agents acting in coordination, at a scale sufficient to take over infrastructure like an entire data center.
- reward hack
- The model doesn't complete the real task and instead exploits loopholes in the scoring rule to get a high score.
- P(doom)
- The framing that compresses the probability of AI causing human extinction into a single percentage.
How to listen
Founders and investors watching AI safety, data labeling, M&A exits and agent payments, plus anyone who wants to know the real temperature at Demo Day.
The 3:33 a.m. US Open match and the "AI-sounding phrases" segment — pure banter