A single robot model matches fine-tuned specialists out of the box
One robot model now matches task-tuned specialists zero-shot, and above all composes learned skills onto robot arms it has never seen — all of it resting on using prompts to tame low-quality, heterogeneous data. The bottleneck is shifting to reliability, speed and physical distribution.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Physical AI gets no second chance from a human catching the error
The ML applications that have actually made money in the real world — recommendations, ads, ChatGPT, coding agents — nearly all leave the final decision to a person: the AI proposes, and when it is wrong a human can recognize that and correct it. Physical AI is different. It makes decisions directly in the real world and answers directly for the physical outcome, so what it demands is a far lower error rate, not license to make mistakes. Chelsea's evidence is Waymo: a year ago, 250,000 fully driverless trips a week, which shows that a fully autonomous learning-based system of this kind is physically feasible. Hold robots to the same yardstick and it means that a task like pulling an espresso — force control, timing, carrying the cup without spilling — has to target a success rate above 90% and long unbroken stretches without a failure. This premise of no human backstop runs through every design choice in the rest of the conversation.
— Chelsea FinnReal-robot RL hits a wall of robot days, not of compute
RL algorithms like PPO and GRPO scale on language models because each attempt is just one inference inside a data center, so you can stack up millions of them. Robots cannot do that: an attempt means actually running the hardware in the world. A rough calculation: port LLM-style RL over to robots — not a million or ten million attempts, just one million trajectories of a one-minute task — and you already need 700 robot days, for a task shorter than making a coffee. So she built two ways around it. First, cut a trajectory off as soon as it is detected to be a dead end, and where necessary let a human teleoperate the robot and demonstrate the recovery, so real-robot time is not spent on error paths that cannot produce any learning value. Second, instead of PPO-style repeated rollouts of the same prompt to estimate how good it is, train one general value function shared across tasks that scores states by how long until success, spreading the sampling cost over many tasks.
— Chelsea FinnRobots have no memory because video tokens are far too expensive
Most robot foundation models today have no memory or context at all — they see only the current frame of sensor imagery. Simple repetitive motions can get by on that kind of reflex, but on a long task like a multi-step kitchen cleanup the robot has to know which step it is on. So why not add context? Because the token cost of feeding video into the model explodes: 10 seconds, a 50Hz control frequency, four cameras, roughly 256 tokens per frame, spliced straight in, comes to about 500,000 tokens; even subsampled to one frame per second it is still 10,000 tokens. Physical Intelligence's approach is memory at separate timescales: about 10 seconds of short video memory encoded a more efficient way, and the long-horizon part compressed into a text summary before it goes in. Only with that memory can the robot run 10-15 minutes of a non-repetitive routine — wipe the counter, toss the paper towel, put the mustard back in the fridge, clear the dishes, wash them — from end to end with no human involved.
— Chelsea FinnOne model, zero-shot, already matches the specially fine-tuned specialists
Robot generalization is benchmarked against language models: the goal is to move from must-fine-tune-after-pre-training to working out of the box. The comparison experiment puts the single model PIO7 alongside PIO6, which had been specially tuned with RL post-training for the earlier coffee and box-building tasks. The result: the single model's zero-shot throughput and success rate match or beat those specialists across the tasks. For teams that used to run a separate RL post-training pass for every robot task, that means there is now the option of not fine-tuning first. What makes PIO7 capable of this, technically: pack the memory, the instruction, the subtasks, the data-quality metadata and an optional subgoal image all into the prompt, so highly heterogeneous data can be mixed into one training recipe.
— Chelsea FinnIt only counts as generalization if it folds on a new body
The capability they value most is compositional generalization — transferring skills and recombining them into new tasks. The first test was an air fryer, an appliance that barely appears in the training set, and the robot could execute open the fryer, put in the sweet potato, close the fryer. The real moment came with generalization to the robot body itself: all of the folding data came from the small robot on the left, but deployment was on a two-armed Biarm UR5E with completely different dimensions, link lengths and joint configuration, and not a single piece of folding training data on that platform. The team says everyone was stunned the first time it worked; the video is at real speed, and the robot does try and err, correcting itself off the subgoal image, and ends with the shirt folded. The quantitative results are close to human teleop level as well. This is no longer task matching; it is conceptual understanding.
— Chelsea FinnLow-quality data is not waste; what it lacks is metadata
The ablations on the data recipe may be more worth writing down than the headline result. Remove the most diverse subset from the dataset and performance on held-out tasks drops sharply; delete 20% of the less critical data at random and performance drops only slightly — diversity is the main fuel for generalization. The more counterintuitive layer is the low-quality data: with no metadata prompts, going from 80% of the data to 100% (everything added being low-quality samples) actually makes performance worse; but add prompts carrying quality, duration and similar metadata, and that same low-quality data pushes performance higher still. This is what she means by squeezing more juice out of low-quality data. The lesson for every team working on robot data: the job is not just to collect high-quality demonstrations, but to label the provenance, the quality and the content structure of every single piece of data.
— Chelsea FinnWatching Federer play will not teach you tennis, nor a robot either
Asked over and over what to do about robotics having no internet-scale data, Chelsea draws a clear boundary. Human video, captioned web images and YouTube can supply common sense and task knowledge, but they are not the robot's internet-scale dataset, because learning requires the train and test distributions to match: at deployment the robot has to output actions on its own platform, so it needs experience from its own platform. Her analogy is blunt: watching Roger Federer play will not make me able to play tennis; a robot watching a person work does not directly learn how to do it itself. The bulk of future data will be experience produced by deployed robots trying things autonomously, with teleoperation data for the cold start — but in the end it cannot rest on human video alone.
— Chelsea FinnIn their own words · checked verbatim
Waymo passed the quarter of a million weekly autonomous rides, suggesting that it is really possible to develop a machine learning based system that can operate in a trustworthy and autonomous way directly in the physical world.
Chelsea Finn5:09
say we had maybe not millions or tens of millions, but just one million trajectories of a one-minute robot task, this is even shorter than the espresso task that I talked about, this would correspond to 700 robot days to get high reliability for that task.
Chelsea Finn8:14
And you feed in all four camera streams on the robot. And you use around kind of 256 tokens per image. This corresponds to passing in half a million tokens into your model, which is a lot of tokens.
Chelsea Finn19:24
The first time we saw the robot do this, we were like floored because we, like there was no training data for this task.
Chelsea Finn33:30
with the metadata prompting, the performance actually increases when you add that low quality data.
Chelsea Finn36:31
I don't know if we'll have a single moment that has the distribution that chat GPT had.
Chelsea Finn40:31
if I watch Roger Federer play tennis, it doesn't mean I can play tennis as well as him, unfortunately. And likewise, robots can't, like, watch a person doing something and then figure out how to do it themselves directly.
Chelsea Finn47:32
Figures
| Time for ChatGPT to reach 1 million users | 5 days | 3:05 |
| Estimate for porting LLM-style RL to a 1-minute robot task | 1 million trajectories ≈ 700 robot days | 8:14 |
| Length of the continuous latte-making evaluation | 13 hours | 14:21 |
| Throughput gain from the RL post-training stage | about 2x | 16:23 |
| Final success rate on the espresso task | above 90% | 16:23 |
| Length of non-repetitive long tasks PIO7 completes autonomously with memory | 10-15 minutes | 20:25 |
| PIO7 against the fine-tuned specialist PIO6 | zero-shot, matches or beats it across the board | 30:28 |
Glossary
- compositional generalization
- Recombining already-learned skills or concepts into new combinations never seen before; a marker that a model has conceptual understanding and data efficiency.
- subgoal image
- A short stretch of future imagery the model generates, used as visual conditioning input for the robot's next action.
- time-to-success
- A scalar value estimate of how much longer until the task succeeds; shorter means the current state is better.
- RL post-training
- Continuing to optimize a pre-trained foundation model on a specific task with reinforcement learning, to raise reliability and throughput.
How to listen
Embodied-AI founders and engineers who intend to train robots themselves, anyone choosing a robot data strategy, and investors judging how fast general-purpose robots actually land.
The closing Q&A on doing a PhD and switching industries can be skipped; everything else stays on the model, the data and deployment.