When everyone can build it, only taste still matters
Everyone can tune models now; picking which one thing is actually worth doing is why PMs matter again.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Models emerge from cooking, not from code
The most counterintuitive part of building product around models, Doshi says, is that you can set targets and tweak training data like you would in traditional product work, yet the result stays partially beyond prediction. She likens it to Snape's potions class: ingredients placed correctly, and magic still erupts unexpectedly. Product work splits in two—half is shaping what you feed the model, half is waiting for training to finish, then finding where it surprised you, and turning that flash into a user-facing experience.
— Tulsee DoshiGoogle's full stack is the edge, not any single model
When pushed on whether Google lags OpenAI and Anthropic, Doshi reframes: Google isn't abandoning frontier models—Gemini 4 Argonne is chasing frontier performance. But her real point is that Google's true advantage is full-stack: chip to model to endpoint products that reach a billion users (she mentions 14 Google products with over a billion monthly active users). This stack lets Google offer different-sized models for different customers—searchers, engineers, network defenders—along that cost-to-quality curve, rather than chasing a single "strongest model" story.
— Tulsee DoshiChain-of-thought monitoring catches misalignment before harm
On AI safety, Doshi skips the "we are careful" platitude and explains the concrete work: Google deploys monitors on models to read their chain of thought in real time, checking whether behavior drifts from assigned intent. Once misalignment appears, they intervene. She's also clear that safety flows both ways—AI hardens defenses. Google partnered with the acquired Wiz on a "Scan for Good" project that helped a hospital system find and patch a security hole, and cases like that shape how she judges "this is moving in the right direction."
— Tulsee DoshiWhen everyone can build, taste becomes the bottleneck
Once AI makes building cheap, Doshi argues, PM value actually rises. With the barrier gone, the scarce skill is picking which of the hundred buildable things is the one worth building. She emphasizes companies like Google, running dozens of product lines in parallel, still need someone to pull order from noise—and once building outpaces coordination, chaos itself becomes the cost. "Taking signal from the noise, aligning everyone on one direction"—that's why PMs are valuable again.
— Tulsee DoshiModel intuition is the gatekeeping hire for product research PMs
These days, hiring for research PM roles increasingly prizes something hard to name: vibes about models. Does this person play these models every day in real scenarios? Can they pressure-test models themselves? Will they dive into data and write their own evals? Doshi's entry advice is specific: don't start big. Begin with five prompts. Even 20 prompts across two models surface clear signals. Get crisp on what "good" looks like at small scale before thinking about infrastructure.
— Tulsee DoshiThe six-level skin-tone standard wasn't designed for global AI
In six years leading responsible AI, one concrete change Doshi pushed through: discovering that Google's early skin-tone evaluation used the Fitzpatrick scale—six levels, originally built for medicine—which inherently biased toward lighter skin tones and couldn't represent the world's actual skin distribution. The team found Harvard professor Monk's 10-level scale, grounded in years of anthropological fieldwork, and swapped it in. That shift now shapes how skin tone renders in Pixel cameras and image generation across products.
— Tulsee DoshiBuild one notch ahead of where models can perform today
Doshi offers a "60-20-20" frame for how far ahead to build: 60% of features run stably now, forming a real, usable prototype; 20% are rough but ship-ready, and you see the model filling the gap soon; the last 20% are "I see a glimmer of light—this would be beautiful if it works." It's more actionable than "ship three months early," since even with models she struggles to predict what's possible in ninety days.
— Tulsee DoshiBurnout is organizational chaos, not hours logged
When anxiety about "we can do more, so we must do more" circulates, Doshi diagnoses the real culprit: not work time itself but two things—churn and chaos inside the org, and doing work that doesn't feel meaningful or rewarding. If you deploy AI right, she expects, it first clears the drudgery, freeing time for the creative, higher-meaning parts. Used correctly, AI tools could ease burnout instead of sharpening it. The catch is using them right: against drag, not as an excuse to do more of everything.
— Tulsee DoshiIn their own words · checked verbatim
it's kind of like, like Snape's potions class, right? So you can put all the ingredients in and you should be thoughtful about what you expect. But you also have to expect that there will be a little bit of magic and that that magic is going to be surprising and that magic is going to be unique.
Tulsee Doshi3:03
deploying monitors that can monitor the model's chain of thought and actually identify whether the model is misaligned with its directed actions or whether it is aligned and then being able to intervene appropriately
Tulsee Doshi10:10
someone who can bring people together and drive consensus and turn chaos into order. I think becomes actually really important because in the absence of that and the ability to build as quickly as we now can, speed can basically introduce chaos
Tulsee Doshi20:19
Realistically, six skin tones cannot capture the diversity of skin tones in the world. Not only that, it was actually built for an entirely different purpose. It was built for like health care purposes.
Tulsee Doshi26:34
what you want to do is actually build such that like 60% of what you've built works now. And your prototype is at least functional, right? Then there's a set of maybe the 20% where you're like, it's not perfect, but it like works well enough.
Tulsee Doshi33:40
I think burnout comes not just from working a lot. It comes from, I think, two places. I think it comes, one, from when you feel like there is churn and chaos. I think that introduces burnout. I think the other thing that introduces burnout is work that you don't enjoy.
Tulsee Doshi45:56
What you should be providing is the insight. What the model should be providing is the data.
Tulsee Doshi48:58
Figures
| Google products with over 1 billion users | 14 | 7:06 |
| Fitzpatrick scale levels | 6 | 26:34 |
| Monk scale levels | 10 | 27:36 |
| Years Doshi led Responsible AI | 6 | 9:09 |
| Stanford flash teams research timing | 12 years ago | 30:39 |
| Doshi's current team work intensity | 150% | 39:46 |
Glossary
- harness
- The system prompt, tools, skills, and context wrapping a model, forming its actual operating environment.
- Pareto curve of cost to quality
- The cost-quality trade-off curve across models; used to match model size to use case.
- chain of thought monitoring
- Real-time access to a model's reasoning process to detect whether its behavior drifts from intent or stays aligned.
- flash teams
- Temporary teams assembled to complete a specific task efficiently, then dissolved.
- Monk skin tone scale
- A 10-level skin-tone scale created by Harvard professor Monk from anthropological research; better represents global skin diversity than earlier 6-level scales.
- Fitzpatrick scale
- A 6-level skin-tone classification originally designed for medical use; misapplied as an AI evaluation standard.
How to listen
Product managers and practitioners who work on model evals, track Google's AI strategy, or want to know what specific PM skills matter in the AI era.
The lightning round starting around 51:03 (books, movies, personal interests) has low information density.