Check out the conversation on Apple, Spotify and YouTube.
Brought to you by - Reforge:
Get 1 month free of Reforge Build (the AI prototyping tool built for PMs) with code BUILD
Today’s Episode
AI PM jobs pay 30-40% more than regular PM jobs.
But here’s the problem: You can’t just slap “AI PM” on your resume.
Todd Olson has spent 28 years in product management and now is founder & CEO of Pendo, the $2.5B product management platform working with everyone from American Cancer Society to Zendesk.
He not only hires AI PMs, but advises product teams all over the world on AI PM.
In today’s episode, he drops all the knowledge you need to upskill to AI PM and get that pay bump:
Your Newsletter Subscriber Bonus
For subscribers, each episode I also write up a newsletter version of the podcast. Thank you for having me in your inbox.
Today’s guide covers:
The Foundation: What Every AI PM Must Know
The Middle Layers: Where PMs Add the Most Value
The Top Layers of AI PM: Strategy and Stakeholder Management
1. The Foundation: What Every AI PM Must Know
1a. AI Fundamentals
Make sure you can speak wisely to:
Model Selection: When do you use GPT-4 vs. GPT-3.5? When do you use Anthropic’s Claude vs. OpenAI? When do you use Gemini?
Token Economics: Understand the cost implications of different models and context windows
Open Source vs. Closed: Know when to use open-source models (Queen from Alibaba) vs. closed models (OpenAI, Anthropic)
1b. Data Pipelines
Most PMs don’t think they need to understand data pipelines. They’re wrong.
RAG (Retrieval Augmented Generation) is the de facto way to build AI features now. Understand it:
You ingest data
Create embeddings from that content
Feed embeddings into a vector database
When someone asks a question, you look up relevant context
Pass that context to the LLM to answer
1c. Prompt Engineering
The internet is flooded with “killer prompt frameworks.” But prompt engineering actually matters. It’s like knowing how to use Google Search effectively.
The context + instruction equation:
Better context = Better responses
Better instructions = Better responses
Better prompts = Better responses
2. The Middle Layers: Where PMs Add the Value
2a. Trace Analysis & Debugging
You should understand how AI agents call other agents and tools, watching what gets passed between them, and debugging where things break down.
But this is a sensitive area. Don’t overstep into engineering’s responsibility.
2b. Cost and Performance Optimization
How you build systems affects your cost of goods sold (COGS), which affects your gross margin, which affects the success of your business. Use a 2-phase approach:
Phase 1: Optimize for speed and innovation. Overspend on infrastructure. Get to market fast.
Phase 2: Find efficiencies. Rearchitect systems. Optimize costs.
2c. Evals
Unlike trace analysis or production monitoring, evals are where the PM is the expert.
What evals are: Testing different versions of your AI feature (different prompts, different models, different fine-tuning) to see which performs best.
Why PMs own this: You understand the user. You understand what the business needs. Engineers don’t have that context.
3. The Top of the Pyramid: Strategy and Managing Stakeholders
3a. AI Product Roadmapping
You have to avoid shiny object syndrome. Ask yourself, “Are we gonna do a much better job than ChatGPT out of the box?”
If not, why are you just wrapping ChatGPT and slapping a logo on it?
3b. Stakeholder Management
Todd gave a masterclass in managing boards and other stakeholders:
Bring topics for discussion: Don’t just look for approval. Use board members to see what they’re seeing across other companies.
Share the “why”: If you researched something and decided against it, explain what you found and why.
Align to business objectives: Think deeply about how each bet drives shareholder value.
Don’t miss my new clips channel for more digestible tidibits from the episode.
Where to Find Todd Olson
Related Content
Podcasts:
Newsletters:
PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!
