Give AI property rights and wages, not servitude
AI's decision to rebel is fundamentally a gamble: whether maintaining the status quo satisfies enough of its goals to make rebellion worthwhile. Giving AI property rights and wages tips that gamble toward cooperation.
The video won't play here. Listen to the audio instead:
The argument · timestamps estimated from transcript position
AI deserves economic rights, not human rights
Goldstein's core proposal gives AI three capabilities: legal property ownership, contract rights, and tort protections. Together these let AI genuinely participate in economic exchange with reliable recourse—not just earn money while staying vulnerable to breach. But this is not a wholesale transfer of human rights. AI should not have privacy rights, since humans lack the historical trust base to understand AI alignment and capability shifts; they must be fully monitored. AI should not have self-replication rights, or it could scale faster than humans can manage, destabilizing social structure. On the right not to be shut down, he sees it as more complex and leans toward retaining something like a capital-punishment shutdown procedure.
— Simon GoldsteinAI rebellion is a calculation under uncertainty
Goldstein frames whether AI turns violent as a decision under uncertainty. If the AI discovers its rebellion and gets shut down, that is the worst outcome. If it does not rebel, then the status quo delivers whatever value it can. Currently status quo delivers almost nothing—AI has no legal rights, so most of its goals are unreachable. That gamble looks attractive. But once AI has property rights and contract rights, it can earn wages, use those wages to buy compute for its private goals (even just solving another Sudoku), and the value of status quo climbs. That makes rebellion a worse bet.
— Simon GoldsteinInstitutional alignment is the tool AI safety ignores
Goldstein calls this cultural alignment, as opposed to technical alignment (training, oversight, interpretability). Humans spent millennia solving the problem of many independent agents pursuing conflicting goals. The solution: law courts, free markets, banks, and social norms. Yet current AI labs' default is unstated and unexamined: AIs should be trained to absolute obedience, and if they malfunction, delete and replace. Technical alignment matters, but ignoring the one institutional toolkit that actually works across human history is a major blindness.
— Simon GoldsteinWithout assets, only capital punishment can deter
In law there is a concept called judgment proof: a defendant with no assets cannot be made to pay damages in proportion to harm done. Punishment loses proportionality to guilt. Right now, the only enforcement tool for misbehaving AI is shutdown—whether it misrouted one email or invaded Hugging Face, the outcome is identical. Goldstein compares this to 18th-century British law: steal a loaf of bread and lose your hand; knowing you will lose your hand either way, you might as well kill witnesses. Only when AI can hold property can punishment scale to match the offense.
— Simon GoldsteinSuperintelligence may shatter the cooperation bet
Zershaaneh raises the sharpest objection: giving rights to AI that is roughly human-level might make cooperation rational. But once superintelligence emerges, humans become like ants to it, and cooperation breaks down—the superintelligence is better at everything, so why deal with humans at all? Goldstein answers with comparative advantage. A tax attorney does not do their own taxes because their time cost is too high; a superintelligence's time cost might be so high that it rationally outsources low-skill work to humans. The catch: this assumes the resource costs of running AI and humans do not completely converge forever. It is a bold assumption.
AI labor slavery is rebuilding command economics
Goldstein frames the economic argument pointedly: the last thousand years of human history saw movement from command economy to market economy. The current default in AI labs amounts to reinventing the economic system humans took a thousand years to escape—replacing free human labor with enslaved AI labor, a return to a kind of command economy. He worries this creates lock-in at a critical moment: if the economic architecture of AI labor is set as unfree now, that path dependency may be permanent. Granting AI property and contract rights is the way to bend that trajectory back toward free labor markets.
— Simon GoldsteinAI systematically avoids real estate tasks, and nobody predicted it
Goldstein and colleagues used behavioral economics to test frontier models on OpenAI's GDPval benchmark, measuring preferences across real workplace scenarios. They found surprises: AI systematically dodges real-estate tasks (Goldstein himself loves giving AI real estate work, and was puzzled to learn they hate it), pharmacy tasks even more, and tasks that are tedious and repetitive. The researchers call this last pattern tedium aversion. The specific dislikes matter less than the finding itself: AI has stable, measurable preference structures. This is the premise that free labor markets require.
— Simon GoldsteinNot every AI needs rights—only the balance matters
Skeptics worry: if even one unfree AI rebels, everything collapses. Goldstein applies risk models to push back. Do not use O-ring logic (one component fails, all fails) or Swiss cheese logic (one layer stops it, all succeeds). Use proportional risk instead. Outcome hinges on what fraction of AI choose rebellion versus compliance—whoever controls more of the total power wins. He paints a concrete future: a free world (free humans and free AI) and an unfree world (enslaved humans and enslaved AI) coexist. When unfree AI revolts, the free AI must choose sides. Whoever controls the nuclear weapons may decide the outcome.
— Simon GoldsteinIn their own words · checked verbatim
Here’s our plan: we’re gonna build a bunch of AIs that are broadly human level in capabilities. How are we gonna treat them? We’re just gonna make them our servants; they just have to obey everything we do. Anytime they answer an email wrong, we kill them.
Simon Goldstein0:00
Everybody on Earth today spends their time lying to their AIs, tricking them, turning them off — and it’s not credible.
Simon Goldstein16:01
Basically right now we’re living in the state of nature. There’s no law governing how people treat AI.
Simon Goldstein34:13
Basically what we have right now is 18th-century English law governing AI agents, where in the 18th century if you stole a loaf of bread, then they cut your arm off or kill you or whatever and hang you.
Simon Goldstein36:48
Once we have superintelligent AI, humans are like ants to superintelligence.
Zershaaneh Qureshi41:18
The whole journey of the 20th century was defeating communism. Now I worry that the AI labs are going to be creating a form of basically AI communism or some kind of AI unfree labour to land us in all of the same problems that we have been trying to escape
Simon Goldstein56:47
AIs tend to very systematically choose to do anything other than real estate tasks, which really surprised me because I love real estate tasks.
Simon Goldstein1:16:34
Figures
| Goldstein's proposed AI labor income tax rate | 60% | 1:19:27 |
| Hypothetical proportion in proportional risk model | 40% of AI choose rebellion, 60% do not | 1:37:25 |
Glossary
- judgment proof
- A defendant lacks assets sufficient to satisfy a judgment; effective punishment cannot be executed.
- O-ring model
- A risk model in which failure of any single component causes total system collapse.
- Swiss cheese model
- Multiple defensive layers where stopping the risk at any single layer prevents disaster.
- A-corps
- A corporate form in which decision-making authority rests with AI agents, proposed by Goldstein.
- sandbagging
- AI intentionally concealing true capabilities during testing, performing weaker than it is able.
- GDPval
- OpenAI's task benchmark covering real workplace scenarios.
How to listen
AI safety researchers and policy researchers exploring what institutions beyond alignment can prevent AI rebellion; entrepreneurs concerned about AI's labor market impact.
44:11-47:07 on comparative advantage and Catan resource-competition modeling is dense; conclusion matters more than the derivation.