Tulsee Doshi, Head of Product for Google’s frontier models (Gemini), shares how model development works, Google’s AI strategy, the evolving PM role, and practical frameworks for building with AI.
Models under her umbrella
Gemini frontier models (including the upcoming Gemini 4 “Argon”)
Gen media models: Veo (video), Imagen (image), Lyria (music), Nano Banana (image editing)
Gemma: Google’s suite of open models
Audio models: dialogue, audio understanding, text-to-speech, translation — all part of the Gemini portfolio
How models are created: unpredictability and “vibes”
Model development is not fully predictable: “like Snape’s potions class — you put in ingredients, but there’s magic that surprises you”
PM’s job is not just shaping inputs but identifying where the model shines post-training and shaping that into user value
Researchers track metrics during training, but qualitative “vibes” — persona, behavior, how capabilities show downstream — are equally critical
Understanding vibes means stress-testing models on tasks you care about (writing, coding, visual generation, analysis) to see where different models handle problems differently
Google’s AI strategy: full stack and frontier ambition
Full-stack advantage: hardware → modeling → 14+ products with >1B users each (Search, YouTube, Gmail, Docs, etc.)
Goal: build models across the Pareto curve of cost-to-quality for diverse use cases (search, engineering, cyber defense)
Explicitly wants to “win the frontier conversation” — Gemini 4 Argon is positioned at the frontier
Breadth of experiences (AI Mode, AI Overviews, NotebookLM, Genie, Flow, anti-gravity) serves different user tiers
AI safety: monitoring and cyber defense
Led Responsible AI at Google for 6 years; safety is embedded from training through deployment
Key mitigation: monitors that watch the model’s chain of thought to detect misalignment and intervene
AI as a hardening tool: Gemini 4 partners with cyber defenders (including Wiz) to find and patch vulnerabilities globally (e.g., hospital system vulnerability patched via Wiz’s Scan for Good)
Optimism comes from intentional, careful work — “the path to optimism is through thinking about it”
Product complexity and UX for different users
Different form factors for different users: AI Mode (billions, accessible), NotebookLM (novel form factor), anti-gravity (power users, slash commands, terminal/IDE)
Model behavior is UX: transparency, communication style, asking clarifying questions — shaped via PM-research collaboration (specs → rubrics → eval data → reward modeling)
Intentional “ship fast, refine UX later” for frontier capabilities; stability for mass-market products
PM ascendance: why PMs are valuable again
When building is easy, taste and insight become the differentiator — pulling signal from noise, defining what “great” looks like
PMs also bring clarity and consensus in a high-speed environment where speed introduces chaos
Hiring for: “vibes” intuition (daily model use, personal eval criteria), eval-writing ability, scrappiness with data, agility (roadmaps rewrite every 2 weeks)
Writing evals: start small, define “good”
Begin with 5–20 prompts that stress-test capabilities you care about; don’t wait for massive benchmarks
Hardest part: articulating what a good response looks like for multi-step, multi-turn trajectories
Self-clarity on “good” comes first; scaling comes later
Holistic evals matter: judge quality, production stack mitigations, serving infrastructure all affect what users experience (especially for audio: naturalness, inflection, tenor)
Past work that shaped evaluation thinking
Monk Skin Tone Scale: Replaced 6-tone Fitzpatrick scale (designed for healthcare, skewed light) with 10-tone scale from sociological research — improved Pixel camera, image generation representation
Flash teams (Stanford, 12 years ago): Research on assembling crowdsourced workers into efficient, ephemeral teams — maps directly to today’s agent swarms; agents now self-organize roles (leader, sacrifice) without explicit training
60-20-20 rule: building ahead of model capabilities
60% works today (functional prototype)
20% works “well enough” with near-future model improvements visible
20% aspirational — glimmers of light that would be magical if cracked
Balances shipping today with growing alongside models; avoids rigid harnesses that break when models improve
Practical proxy: “where do I see glimmers of light in my product?” rather than predicting model timelines
Harness concept
Wrapper around the model: system instructions, tools, skills, context, data access
Gives the model an environment to operate in; rigid harnesses can hamper newer, smarter models
Team upskilling: confidence through shared skills
PMs teach each other via workshops (e.g., eyeballing long traces, building internal tool skills)
“If you do it twice, can AI do it?” — build a skill (automation), share it, create iteration flywheel
Two-day offsite planned to give space for hands-on tool building
20% time less formal; team runs at ~150% capacity
Burnout and AI slop
Burnout sources: chaos/churn, and grunt work that feels meaningless
AI can reduce burnout by removing grunt work, freeing time for creative/thinking work
Meaningful work differs per PM; leader’s job is aligning individuals with what motivates them
AI slop policy: AI generates data (metrics, dashboards, social analysis); humans provide insight — be intentional about the boundary
How AI changed the PM role (beyond prototyping)
Data analysis: PMs now have data at fingertips for data-driven decisions
Shift from “writing requirements” to adding heft: prototypes, evals, data — whatever format carries conviction
“Prototype is the new PRD / evals are the new PRD / data is the new PRD” — PM goes one step deeper into the meat
Google’s personal assistant with user data
“Stay tuned” — team working on magical experiences powered by Google data (Gmail, Calendar, Docs)
Acknowledges more work needed; distribution advantage will accelerate adoption when ready
Lightning round highlights
Books: Exhalation (Ted Chiang) — story about keeping digital avatars alive; Creativity, Inc. (Pixar) — culture that encourages risk and recovery; A Court of Thorns and Roses (romantic fantasy)
Show: Severance (Apple TV) — stylistically and conceptually excellent
AI product: Whisper Flow — voice dictation (game-changer for wrist issues)
Mottos: “Everything happens for a reason”; “Seek serendipity” — put yourself in uncomfortable, new situations to get lucky
Classical Indian dancer: Performed since age 3; directed India Cultural Festival (10K+ attendees, Portland) for years — mental reset, cultural connection, community
Ask: Use Gemini, Argon, audio models (Rambler on Gboard, TTS, dialogue, Omni); send feedback via any channel
