Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by: LogRocket

This is the AI analytics, logs, and session replay tool you need. I ran a head-to-head eval against PostHog to test their AI, and published my results here.

Check it out

Today’s episode

There are hundreds of guides on writing PRDs with Claude. Dozens on running user interviews. Almost nothing on how a VP of product actually uses it in-depth.

Matt Wensing is the VP of Product and Design at Customer.io. They crossed $100M ARR, just shipped an AI agent, and are one of the fastest growing companies in B2B SaaS right now. I asked him to show me his actual documents, his actual Slack threads, and the exact sessions where Claude helped him produce leadership grade output.

What he showed me changed how I think about AI for leaders. Claude has the instincts of a brilliant new hire, it wants to deliver before it fully understands what you need, and at the VP level that gap shows up fast.

Matt has spent months figuring out how to manage it, and in this episode he shows you everything. You’re not going to find another video in the world with real VP of product examples from their real work.

Apple Podcast

Spotify

• To get access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

• To get access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

Newsletter Deep Dive

Thank you for having me in your inbox. Here is how a VP of product at a $100M ARR company actually uses Claude:

1. How a VP of Product Actually Works With Claude

   • Why Claude produces slop, and why executives catch it instantly

   • How to build leadership grade presentations with Claude

   • Using Claude as a thinking partner vs. output machine

2. A VP of Product’s AI stack

Subscribe now

Before we get to the longer article, here’s the most important takeaway in an infographic.

Share

Now into the deep dive…

1. How a VP of Product Actually Works With Claude

Why Claude produces slop, and why executives catch it instantly

Matt has a rule he shares with every new hire at Customer.io:

Junior employees, however talented, will race to the finish line before they have enough context to do the job well. They hear the first instruction, want to impress, and just go.

They skip the clarifying questions. They come back with something that looks complete but misses the point entirely, and now you have to tear it down and start over.

Claude does the exact same thing.

Give it a goal and it will sprint. It will generate a low grade Word doc. It will use jargon mid session and put it straight into the executive document. It will suggest the next step before you have finished thinking through the current one, nudging you toward a deliverable you are not ready for yet.

Matt calls this “the slop problem.” And it gets worse the higher up you go, because the audience reading your output gets sharper.

Executives are the best slop filters in the room

Senior leaders spend their careers reading documents that try to persuade them. They have a filter that kicks in within the first paragraph. They are not looking for polish. They are looking for evidence that the person who wrote this actually understands the multi dimensional nature of the problem.

A one shot alignment deck fails that test every time. Years of organizational history and unspoken context get compressed into a clean seven slide narrative. Matt calls this the “flat projection problem.” You present a shadow of the problem to people who have been staring at the real thing for months. They filter it out, and depending on where you sit in the hierarchy, that looks like polite nodding or your work simply getting ignored.

The fix is a different way of working with Claude entirely.

Instead of giving it the full problem upfront, you feed it context in layers, one piece at a time, making it earn the right to generate the final output. You kill the eager suggestions the moment they show up. You stay in control of the pace.

Matt puts it simply - a 200 iteration session with a great deliverable at the end is worth far more than saying yes to the first draft.

The document you produce is less polished but higher in insight density. When an executive reads something that matches the complexity they have been living with, they do not filter it out. They lean in.

How to build leadership grade presentations with Claude

In the episode, Matt shared an interesting story, he woke up at 5am with a company all hands due at 11. His week had been fully booked, the way every leadership week is (you know this better).

He started with - Six hours, a blank slide deck, and Claude.

By 11am, he had a polished Q2 roadmap presentation in front of 400 employees.

Here is exactly how he built it.

Step 1: Take inventory before you touch Claude

The single biggest mistake leaders make is jumping straight into building. Matt’s first move was not opening Claude. It was taking stock of what raw material he already had.

In this case he had two things. A Zoom recording of the engineering team’s demo day, and a strategy doc outlining Customer.io‘s three investment themes for the year. Neither was shaped correctly for the audience he needed to reach. The demo day was engineering talking to engineering. The strategy doc was internal direction, not a story for 400 people across sales, marketing, and engineering.

Raw ingredients first. Always.

Step 2: Pivot, don’t write

Once Matt had his ingredients, he did something specific. He did not ask Claude to write slides. He asked it to reorganize the demo day Zoom transcript around the three themes from the strategy doc.

Think of it as matrix multiplication. You have content shaped one way. You have a framework shaped another way. You ask Claude to pivot the first into the shape of the second. The output is not a finished product. It is strategically shaped raw material you can actually work with.

Step 3: Using Claude as Script Writer

Matt is a believer in show don’t tell. So he built the slides before he wrote a single word of the talk track.

Once the slides were done, he took screenshots of each finished slide and fed them back into the same Claude session. Then he asked Claude to write the talk track using all the context it had already ingested, the Zoom transcript, the strategy doc, the slides themselves, with one specific instruction. Do not just repeat what is on the slide. Use everything you know to say something more interesting than what is already there.

The result was a talk track that added depth instead of narrating the obvious.

Using Claude as a thinking partner vs. output machine

Matt does a lot of his Claude sessions by voice, on walks. You will see why this detail matters in a moment.

He was trying to build a pricing philosophy document for his CEO. The kind of document that shapes how an entire company thinks about value and growth for the next two years.

He did not tell Claude any of that upfront.

Start in the abstract, reveal the domain last

If he had opened with “help me build a pricing philosophy for Customer.io,” Claude would have pattern matched against every pricing philosophy document in its training data and produced something that looks like all of them. Generic tiers. Value metrics. A document that could have been written for any SaaS company.

So Matt tricked it.

He opened with a biology metaphor. An ecosystem. A two by two grid. Things enter from one corner, move through it, sometimes grow, sometimes stall, sometimes leave. He asked Claude to think about the different paths something could take through this system.

If you can see, he talked in completely abstract terms, never mentioning customers or pricing.

Claude had no idea where he was going. And that was exactly the point.

Here's the exact prompt he tried, transcribed straight from voice mode, stumbles and all:

Layer complexity in slowly

Matt did not dump everything into the session at once. He started with the simplest version of the framework, four boxes, basic rules of movement between them. Once Claude understood that, he added one layer of complexity. Then another and another.

He compares over prompting to a bad game night host. The person who explains every rule and every exception before anyone has touched a card. Nobody understands the goal. Nobody knows how to win.

Feed Claude the core rules first. Let it stabilize on those. Then add the exceptions.

Reveal the domain only when the model is clean

It was not until deep into the session, many iterations in, that Matt finally told Claude what the exercise was actually for. A pricing philosophy document for his CEO.

You have spent all that time building a clean mental model in the abstract. The moment you reveal the domain, you can apply that model as a stress test. Does it generate things that match reality? Does the framework hold up when pointed at actual customers? If yes, you have something worth presenting to a CEO.

2. Customer.io’s full weekly AI stack

Most of what you have read so far is about Claude desktop. Long sessions. Iterative context loading. Abstract frameworks that get applied to real problems. That is the deep work layer of Matt’s stack.

But there is a whole other layer running underneath it, every day, without Matt having to think about it.

The analysis bot

His team built an internal Slack bot with access to Snowflake. Matt had 2,000 customer records he needed to analyze for a report going to other executives. Instead of waiting for a data pull or writing SQL himself, he talked to the bot in natural language. Asked questions. Got answers. Followed up. Went deeper.

Two things he is careful about here. One, he has a data team that can jump in when the bot produces something that does not look right. Two, he never takes a non-deterministic answer at face value for something going into an executive report.

The Slack scanner

Customer.io has over 400 employees spread across dozens of Slack channels. There is no version of Matt’s job where he can read every thread that might need a product perspective.

So one of his teammates built a scanner. It runs across several dozen channels continuously, looking for conversations where a product person should probably be involved but is not. It surfaces those threads to Matt at set times during the day, without overwhelming him.

Matt is clear about what this tool is and is not. It keeps him close to the ground while he is deep in a Claude session or running back to back executive meetings. He can see a support conversation with a specific customer that needs product input, flag it, tag the right person, and turn it into a process improvement opportunity all in the same thread.

This is the thing most leaders lose as they get more senior. Proximity to the real problems. The scanner gives it back without adding hours to the day.

Chiefys

The third tool is the most strategic one. Chiefys is a bot that Customer.io‘s CEO Colin built. It lives in Slack and has one job. It holds the corpus of Customer.io‘s official company documents, the operating model, the strategy docs, the things the company actually runs on, and it checks new work against all of them.

When Matt produces something new, like a pricing philosophy, he can run it through Chiefys. It will tell him whether what he just wrote contradicts anything already established across the company’s core documents.

It works in both directions. New document contradicts an old one, Chiefys flags it. Old documents go stale because the company shipped something new, Chiefys flags those too. Every leader has experienced the pain of getting alignment on something in January and watching that document become invisible by April because three other things changed around it. Chiefys is the accountability check that nobody has time to do manually.

Get Transcript

Where to find Matthew Wensing

• 1:1 Video Consultation

• LinkedIn

• X

• Customer.io

Related content

Podcasts

1. PM’s Guide to Claude - YouTube | Spotify | Apple

2. How to Become a Builder PM - YouTube | Spotify | Apple

3. We Built an AI Product Manager in 58 mins - YouTube | Spotify | Apple

Newsletters

1. How to Build Product Strategy with Claude Code

2. My Claude Code PM OS

3. Team OS

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Refer a friend

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Bolt.new - Ship AI-powered products 10x faster

2. Product Faculty - Get $550 off their #1 AI PM Certification with code AAKASH550C7

3. Customer.io - Send smarter messages using your product data

4. Ariso - Ship AI agents and features faster, with fewer regressions

5. Jira Product Discovery - Plan with purpose, ship with confidence

Today’s episode

Six months ago, I showed you Codex is the best way to use ChatGPT for PM work.

Those who tried it have had great reviews:

But most of the content out there for PMs is all about Claude Code (myself included). So I wanted to get you the advanced setup for Codex, that’s up to date as of June 2026.

That’s what today’s episode is - an inside look at expert usage of Codex, from an OpenAI PM himself.

Abhi Muchhal is an International Growth PM at OpenAI and before that was at Meta and Nubank. He opened his actual setup on camera: the harness… everything. Then he covered how to crack a role like OpenAI yourself.

Apple Podcast

Spotify

• To get access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

• To get access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

Newsletter Deep Dive

Thank you for having me in your inbox. Here is the complete guide to using Codex like a senior PM:

1. The PM’s Ultimate Guide to Codex

   • The Codex harness setup

   • Automations that run while you sleep

   • Prototyping without a designer or engineer

2. How ChatGPT has Grown to 1B monthly active users

3. [BONUS] How to break into AI PM at a company like OpenAI

Subscribe now

1. The PM’s Ultimate Guide to Codex

In this section, we cover:

• How to setup Codex or your agent’s harness

• The 3 automations Abhi swears by

• How to build prototypes in Codex

The Codex harness setup

As I wrote in my Ultimate Guide to ChatGPT Codex, the harness is what separates a one off interaction from a persistent system. It is the connectors, the folder structure, the permissions model. Without it, you are just using a very expensive autocomplete.

Data connectors

The first thing to set up is every data source Codex needs to know about.

At OpenAI, the international growth team was pulling from seven or eight sources. Tableau dashboards. Databricks dashboards. Different tools, cadences, and formats.

Connect all of them.

The permissions model

Give Codex three levels of permission:

1. Reading tasks get full autonomy.

2. Synthesis and writing drafts get full autonomy

3. Anything going to another human needs your approval.

Skills, skills, skills

One growth engineering team at OpenAI got tired of running experiment reviews by hand. So they built a skill. Point it at a StatSig experiment, and it writes the hypothesis, monitors the experiment, and generates a postmortem and recommendation automatically.

That is what skills are. Reusable workflows, written once, triggered by name. PMs skip this layer entirely because building a skill feels like an engineering task. It is not.

When asked who builds these skills, Abhi said:

“The beautiful thing about Codex is that the person who cares the most is the one who makes the skill. It doesn’t matter if it’s an engineer, an analyst, or even a PM. I’ve made some skills as well.”

I’ve been hammering this home for months. Good skills are everything!

Abhi’s Top 3 Automations

Abhi swears by three automations:

Automation 1 - Daily Slack triage

Abhi works across time zones. By the time he wakes up, his Slack has 200 unread messages across a dozen channels. He was missing important things because there’s no priority filter for him.

So he built a Slack inbox triage that runs every day at a fixed time. It knows which channels matter, which senders are priority, and what kinds of messages need a response. It delivers detailed outline, things you have not read that you should, and things you have not replied to that are waiting.

The key prompt structure:

Automation 2 - A dashboard that updates itself

Abhi’s team was pulling from seven or eight different sources to understand how ChatGPT was growing across priority markets.

So he built a web app that pulls from all of them, refreshes every morning at 9:30 AM, and gives you the combined view plus the key takeaway. Country tabs. Top line metrics. Codex generated strengths and risks for each market relative to peer countries.

Let’s take a moment here and ask yourself - what are the sources you check manually every week, and what would it look like if they were in one place with a machine generated TLDR waiting for you?

Automation 3 - A weekly update

The weekly stakeholder update is the most time consuming, least value added task on most PM calendars. It pulls from Slack, Google Drive, Notion, and the same dashboards powering the growth view.

So he automated the synthesis. Codex pulls from every source, writes the first draft, and posts it to a channel he owns. He reviews it, edits the things that need his judgment, and sends it to his stakeholder group.

Prototyping without a designer or engineer

Before AI, getting a prototype in front of engineers meant writing a PRD, convincing a designer to prioritize your request, doing two or three rounds of mocks, and then finally handing something over. That whole loop could take three weeks for a simple feature. OpenAI cuts the loop, as Abhi described it:

Step 1 - Replace the PRD with a prototype plus companion doc

A prototype with a companion doc that addresses the obvious questions gets to the same place faster and creates a better conversation.

The companion doc answers ten questions. Why are we doing this. What must work for V1. What are the edge cases. What does success look like. It’s more like an FAQ that lives beside the prototype.

Showing a working prototype changes the quality of the conversation. Engineers stop asking what you want and start asking how to make it better. That shift leads to a huge leap in productivity.

Step 2 - Spec the output before you prompt

Vague prompts return vague prototypes. Before you send a single message, write down three things:

Paste that as your first message. Everything Codex builds follows from that. The more specific the spec, the less time you spend in correction loops.

Step 3 - Preview in app before handing off

Codex can now spin up a local preview inside the interface. You do not need to open a browser. You do not need to configure localhost.

Run the preview. Check the layout. Ask Codex to match a specific brand aesthetic if needed. Use Playwright to verify the fix. Then hand it to engineering team.

The biggest failure mode at this stage is showing an engineer a prototype that crashes in the first click. Test it before you show it. Codex will catch most of the surface level issues itself.

Getting to 80% without engineering time is non negotiable.

2. How ChatGPT has Grown to 1B monthly active users

There is an assumption baked into every AI PM playbook written in the last two years. The user has a desk. The user reads English. The user has a knowledge worker job that involves staring at a screen.

Abhi’s job is to break that assumption.

As International Growth PM at OpenAI, he is responsible for markets where knowledge workers are a single-digit percentage of the working population. India is under 10%. Brazil is 10-20%. These are OpenAI’s fastest-growing markets.

He emphasized three strategies the team implemented that I want you to notice:

Building for language diversity

The people in Bangalore opening a bookstore are not thinking in English. They want a website in Kannada. They want invoice templates in Hindi. They want a YT shorts caption that lands for an audience in Tamil Nadu.

Codex can generate, translate, and format across languages at a level that was not possible 18 months ago. Character rendering in Indian scripts, Japanese manga-style layouts, multilingual infographics.

Synthesizing information that lives on WhatsApp

In markets like India and Brazil, business happens on WhatsApp.

With checking everything manually you can now point Codex at your WhatsApp desktop app, ask it to summarize what you missed, identify action items, and draft a reply based on your calendar availability. In the episode, this entire flow took just over a minute. The reply appeared in the WhatsApp composer, pre-typed, waiting for you to hit send.

This is a workflow every PM building for international markets should understand.

Validating decisions in markets you are not in

The hardest part of international growth PM work is not building the product. It is understanding whether a feature that works in the US will land in Southeast Asia or West Africa.

Codex cannot replace user research. But it can synthesize the data you already have and surface the gaps faster. In growth work, the teams running the fastest are not collecting more data. They are synthesizing what they have with more precision.

For regulated markets or markets where your data pipeline is limited, the synthesis layer matters even more. If your Databricks dashboards are behind a compliance firewall, a Codex skill that only reads from approved exports still beats manual review by hours.

The PM who understands where their product actually lives in the world is harder to replace than the PM who only knows where it is built.

3. How to break into AI PM at a company like OpenAI

The question I get most from PMs reading this newsletter is some version of - how do I get from where I am to OpenAI, Anthropic, or a frontier lab?

I wrote a complete guide on breaking into OpenAI here. Since then, I have talked to four OpenAI PMs on this podcast.

Three key ingredients come through from the 4:

Key Ingredient 1 - Deep PM fundamentals, not just AI familiarity

Every frontier lab PM I have talked to had a serious pre AI career. Not just a familiarity with the tools. Deep PM fundamentals. Structured thinking. Analytical decision making. Communication under ambiguity.

This should be reassuring. The PM skills you have built are not obsolete. They are the foundation. The AI fluency is what you build on top.

What I covered in my AI PM job search guide still holds. The comp at frontier labs is real, but the evaluation criteria are not that different from a strong senior PM role anywhere. You need to show you can own a problem end to end.

Key Ingredient 2 - You have to build something

Every OpenAI PM I have spoken to built something with the APIs before they were hired.

Not a side project that exists as a GitHub repo nobody has seen. Something that actually ran. Something you had to debug. Something where you discovered what Codex or GPT could not do and had to work around it.

The builder credential matters because it is the only way to know what the model actually fails at. You cannot interview your way into that knowledge. In Abhi’s case he built a Chrome extension for real time language translation, deployed it on the OpenAI API, and was demoing it at the time of their application.

One honest caveat here. This path is more accessible to PMs who have slack time to build and who can access paid API credits. If you are in a market where that is not trivially affordable, start smaller. A Codex CLI project on your own machine costs almost nothing to run. The goal is to understand failure modes, not to ship a production app.

Key Ingredient 3 - Speak the language of evals

This is the one that surprises most PMs.

Evals are how frontier labs measure progress. A rubric that defines what good looks like for a specific capability, a baseline measurement, and a goal to beat.

You do not need to have run 50 evals to talk about this fluently. You need to understand why they exist, what they replace, and what a good eval measures versus a bad one. I covered this in depth in my LLM Judge guide and in the Ankur Goyal episode where we built one live on camera.

The PMs who get the farthest in frontier lab interviews are the ones who can say: here is a capability I care about, here is how I would measure it, here is how I would know if the model improved.

Every PM who can think in evals is already speaking the language of the companies building the future. That’s it for today, try to reread and understand the full workflow. See you in the next episode.

Get Transcript

Bonus. How to Get the Most Out of Codex:

Share

Where to find Abhi Muchhal

• LinkedIn

• X

Related content

Podcasts:

1. The Ultimate Guide to ChatGPT Codex

2. How PMs Ship 100K Lines of Code at OpenAI

3. Evals are the new PRD

Newsletters:

1. My PM OS

2. AI Agents Guide for PMs

3. How to Land a $300K+ AI PM Job

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Product Faculty - Get $550 off their AI PM Cerficiate with code AAKASH550C7

2. Bolt - Ship AI-powered products 10x faster

3. Customer.io - Send smarter messages using your product data

4. Ariso - Ship AI agents and features faster, with fewer regressions

5. Pendo - The #1 software experience management platform

Today’s episode

Most companies are still debating whether PMs should ship code. OpenAI is debating the best ways for PMs to ship code. They’re living in the future.

The builder behind a lot of that harness engineering is Ryan Lopopolo. He wrote the OpenAI post on harness engineering and runs a frontier team where PMs, designers, and engineers all ship using the same system.

The wild part for me? His PMs shipped around 100K lines of production code.

Did they open the IDE? Hell no! Their coding happened through PRDs, tests, docs, and harness rules. The model did the typing.

As someone who spent a decade in PM growth roles, I’ve seen how long it takes to move a feature from PRD in a doc to code in prod. For most companies, that latency is weeks, months, or quarters.

In Ryan’s world, it can be days, hours, even minutes. And the PM is inside the loop - not watching from Jira. So I had to understand how that works - and that’s today’s episode:

Apple Podcast

Spotify

• If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

• If you want access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

Today’s post

Here’s the guide to shipping code like PMs at OpenAI:

1. How the harness works

2. Ryan’s Frontier team case study

3. The new job for PMs, designers, and engineers

Subscribe now

1. How the harness works

You don’t need to copy every detail of OpenAI’s setup.

You do need the big pieces, because that’s the system you’ll be dropped into as an AI PM. Ryan thinks of the harness as the environment inside the repo that teaches the agent how your team builds software.

agents.md and the docs tree

Every repo has an agents.md file at the root. When they run Codex on that repo, this file is always in the model’s context. agents.md does two things. It explains the operating loop like read docs, plan, implement, run tests, and ask for review. Then it points the agent at the docs tree like markdown files with performance patterns, networking practices, user journeys, design guidelines, and past decisions.

Each run’s execution plan gets written back as implementation history. Design docs move out of slides and Slack into the repo. And tests fail if docs fall out of sync, which forces the agent to update them while editing code. The model needs a repo that it can search to show how your team works. As a PM, that’s where your decisions belong.

Tests and lints that enlist taste

Ryan’s team encodes taste and non‑functional requirements directly into tests and lints. No please remember I beg of you paragraphs. A few examples:

1. The build fails if any user‑facing markdown or HTML string uses straight quotes instead of curly quotes, because their design team is finicky about typography.

2. Business logic has to belong in the right modules. Tests enforce boundaries so a ball of mud doesn’t form.

3. When docs need to move together, tests enforce that they stay in sync.

Because the model is trained to make tests pass, each failure message is an instruction. You’re not arguing taste in Slack. The suite is the style guide. In growth roles, I learned the hard way that if something matters, you have to measure it and enforce it. Their approach is the same idea, just pointed at code and docs instead of funnels.

Review agents as virtual staff engineers

On the review side, they run a matrixed CI job that spins up specialized review agents, each with a persona doc:

1. frontendarchitect.md

2. reliabilityengineer.md

3. appsecengineer.md

That feedback goes back into the loop. The implementation agent uses it to fix the current change. The team updates the docs or tests so the same class of mistake doesn’t come back.

Observability and UI control

Finally, the harness gives the model eyes on the product. The agent can spin up a local observability stack (metrics, logs) and use it to debug like a human. With GPT‑5.5’s computer use, it can click through the app, inspect elements, and verify flows end‑to‑end.

The rule is simple; the agent has to prove the feature works through the same paths your users take. As a PM, that’s where your acceptance criteria also need to land.

2. Ryan’s Frontier Team Case Study

In mid‑2025, his team ran an experiment. Could Codex do the entire software engineering job for a new internal agent product?

The constraints?

1. Start from an empty repository.

2. Build an app that supports non‑engineering knowledge work inside OpenAI.

3. Let the agent handle on‑call triage, internal tools, and text‑heavy workflows.

4. No human is allowed to type production code. Engineers can only touch the harness.

By the end, the app had around 1M lines of code. It also had roughly 250K lines of markdown prompts inside the repo. And it ran as an Electron app with multiple agents coordinating summarization, repo gardening, skills distillation, and execution.

Any time the agent failed, the team asked what was missing in the harness that led to this failure. The codebase improved because humans improved the environment. Nobody jumped in and rewrote everything.

3. The new job for PMs, designers, and engineers

For most of my 10+ years in product, PMs were kept away from the repo. You were the human API between customers and engineering. You wrote PRDs, argued in sprint planning, and waited. The typing was someone else’s job.

Ryan describes a different world.

In his team, code is no longer the scarce resource. With GPT‑5, the model can generate code at scale. The difficult part is validating it, safely deploying it, and making sure it solves the right user problem. That means the harness (the environment of docs, tests, lints, review rules, and observability) is where work gets done.

Once the harness is strong, engineers aren’t the only ones who can write code. PMs and designers easily plug into the same system.

And literally everyone gets a new job description.

The new role for PMs

In late 2025, Ryan’s team wanted to explore a ‘skills system‘ so the agent could learn and reuse user preferences for data analysis. In most companies, this would kick off a chain of product reviews and engineering estimates. Here is what they did instead:

1. The PM wrote a markdown PRD that defined the skills library

   • How the agent should interview users about the metrics

   • How should it store and reuse that knowledge

   • How this should appear in the product surface

2. The team reviewed the PRD once in their weekly meeting.

3. By the end of the week, the feature existed in the app. The tests the PM had vibed into existence were passing.

The PM and engineers did not need a week of back‑and‑forth. The harness carried the intent into code. This worked because business logic was embedded in modules with high‑fidelity fakes. Also, the tests from the product side exercised the behavior.

The important bit: the PM’s leverage came from writing artifacts that the harness understands. If you’re a PM reading this, imagine your next feature:

1. Could you express it as PRD + tests + evals the agent can run on its own?

2. If you couldn’t touch the IDE at all, what would you have to write differently?

That’s the muscle you need to build.

Designers generate painted doors with

Early on, if a designer tried to ship a feature, they had to schedule agent runs inside their internal app. Then, to demo it quickly, they had to push a backend‑style cron system into the front of the stack. Usually, the scheduler logic ended up tangled inside frontend JavaScript.

Ryan looked at it and knew they’d regret it later. So decided to revert the change. Their team came out of that with a simple norm:

1. Designers own full painted‑door flows at the front of the app: the UI, the interaction pattern, and how scheduling is exposed.

2. Behind the door, the backend can be a no‑op plus instrumentation. They measure clicks, intent, and drop‑off before committing engineering time.

The harness then decides, based on data, which doors deserve a backend. So the JD for designers is to build high‑taste, high‑signal surfaces that the harness sees as experiments. A designer now must give the agent and engineers the best possible signal about what’s worth building.

Coding engineers became harness engineers

Ryan’s definition of the modern engineering job is very close to how I think about great growth engineers. Your value is the leverage you create for everyone else.

In his team, the primary output is leverage. Every engineer is responsible for adding tests, docs, CI rules, and observability that let many concurrent agents work. Internally, they use a simple mental model. Think of it as you just hired a cluster of interns named Codex. Your job is to manage them.

Emotionally, that’s difficult. Many engineers (including Ryan) grew up measuring their contribution in commits and lines of code. Letting the model do the typing can feel like losing the craft. But he argues that the craft moved. The new skill now is who can build the harness that makes GPT‑5 a strong teammate for this particular product and company?

“Every engineer on the planet now has hundreds or thousands of concurrent hands on keyboards, modulo token budget.”

That’s the world your PM career is walking into. You’ll be working with engineers who measure themselves by the systems they build.

Why should YOU care? Does it matter?

Most teams use AI to speed up function‑level coding but keep the rest of their software development lifecycle unchanged. Ryan’s world is what it looks like when you redesign the system around agents that can do the full job. The differentiator is how much of your team’s judgment is embedded in your harness.

If you’re a founder or product leader, that’s what AI strategy now means. It’s how you design the engineering system your PMs and designers plug into, so they can ship.

If you’re a PM, this is the future job description. You’ll still need taste, discovery skills, and stakeholder management. You’ll also need to think like a harness engineer: what tests, docs, and rules would I write if the model were my team?

Get Transcript

[Bonus] Downloadable Takeaway

I turned Ryan’s six‑month harness plan into a one‑pager you can use as a checklist with your team.

Where to find Ryan Lapopolo

• X

• LinkedIn

• OpenAI

Related content

Podcasts:

1. How to Run Evals in Claude Code with Aparna Dhinakaran

2. How to Build a Full AI Dev Team in Claude Code with Gabor Mayer

3. This CPO Uses Claude Code to Run His Entire Work Life with Dave Killeen

Newsletters:

1. PM’s Guide to Claude with Pawel Huryn

2. How to Become a Builder PM with Mahesh Yadav

3. How to Build a Team OS in Claude Code with Hannah Stulberg

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Superhuman - The fastest email experience ever. Get 1-month free with my link

2. Land PM Job - My 12-week AI PM + Job Search Course

3. Vanta - Automate your compliance. Close deals faster

4. Product Faculty - Get $550 off their AI PM Certification, code AAKASH550C7

5. Bolt.new - Ship AI-powered products 10x faster

Today’s episode

If you’re building in Claude Code, you should learn how to build evals in… Claude Code.

There are two major evals platforms in the market today. We’ve already had the CEO and founder of one on, Ankur Goyal of Braintrust.

Today we return with the other major platform, Arize. And we have on the CPO and founder, Aparna Dhinakaran.

Many of the smartest AI teams are running their evals on Arize - Uber, Booking.com, Pepsi... I even offer Arize in my bundle. So now we bring you the guide to using it.

This episode builds upon the basics I taught with Hamel Husain, Shreya Shankar, and Ankit Shukla. It even includes a segment that, if a candidate did it in an interview, Aparna said she would hire them on the spot:

Apple Podcast

Spotify

• If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

• If you want access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

Newsletter Deep Dive

I’ve put together the complete guide for running evals in Claude Code.

1. Evals don’t have to take forever and be hard

2. Step-by-step, how to run Claude Code evals

3. How to build a self improving loop

4. The new PM operating system

Subscribe now

Save this. A 3-step playbook on how to run Claude Code evals with exact commands.

Share

1. Evals don’t have to take forever and be hard

In the past, I’ve taught you the ways of Hamel Husain and Shreya Shankar who said that you need to look at the traces yourself, categorize them yourself.

In practice, what this means is a lot of PMs skip evals because it takes too long.

That’s a mistake. You can actually use Claude Code to do all those steps.

Here is what changed.

The old assumption - Evals need human-led error analysis

The old model assumed you had to be the one reading every trace. You needed to understand what went wrong before you could write an eval that tests for it. That meant the eval quality was capped by how much time you had.

That assumption made sense when models were weaker. It does not hold the same way anymore.

The new reality - Claude can suggest your first eval

When I asked Aparna, she said:

“I think it’s okay to start with Claude suggesting what a good suggestion of an eval could be. These models have gotten so good. Having it go through and look at your answers and suggest, that probably is something you should flag and look at. I would trust it. I would trust it as a first pass.”

This is not about replacing rigorous eval work. Hamel and Shreya’s methods still produce the most calibrated evals over time. But you do not need to start there. Claude gives you a working v0 in minutes. That v0 gets you data. That data gets you something to actually improve.

Aparna demoed “vibe based evals” using Claude Code. Here’s how it works.

2. Step-by-step, how to run Claude Code evals

You have an agent running. It is producing output. You have no idea if that output is accurate or just confident looking.

Here is exactly how to change that in one session.

Step 1 - Install the Arize skills into Claude Code

Open Claude Code. Run one command:

Claude Code now has access to instrumentation, eval suggestion, and improvement skills, you do not need to write any eval code yourself. You just ask Claude Code in plain English what you want. It calls the right skill for the job automatically.

The instrumentation skill is the first one you need. It looks at your codebase, identifies every LLM call and tool call, and wires them to Arize for tracing.

Step 2 - Ask Claude to suggest an eval

Once you have traces, ask:

Claude looks across your traces and returns candidates. In Aparna’s live demo it surfaced three:

• Report groundedness. Do the issues cited in the final PM report actually exist in the source data?

• Priority alignment. Does the P0/P1 ranking in the report match the highest-scored issues?

• Report actionability. Is the output something a PM can act on the same day?

Good starting point. But these are output level evals. You want to go one layer deeper.

Step 3 - Get specific about what you actually care about

Ask something more targeted:

Claude runs this across your spans. In the demo it flagged four failure categories: bug priority scoring, feature request scoring, legacy scoring system conflicts, and low priority edge cases.

Now you have something real. Specific categories of failure, with specific spans attached. That is what you need to improve the agent systematically.

You cannot fix what you cannot name. The eval names it for you.

3. How to build a self improvement loop

Your eval is running. It is flagging failures. The natural next question is: now what?

Most PMs stop at the eval. They have the signal. They do not build the system that acts on it. That is the difference between a PM who runs evals and a PM who has a self-improving agent.

The loop - three moves, on a schedule

The loop is not complicated. It is three moves that run automatically on a cadence you set.

Move 1 - Fetch everything the eval flagged as wrong. Claude pulls every span from the last cycle where the priority accuracy eval fired.

Move 2 - Find the pattern. Claude groups those failures by category. Bugs being underweighted. Long feature requests getting inflated scores. Edge cases involving third party integrations always landing wrong. The categories tell you where the agent’s logic is broken.

Move 3 - Propose a fix. Claude suggests a specific prompt change or scoring adjustment that would address the most common failure category. A concrete edit you can review and approve.

To set this on a schedule, ask Claude:

Claude spins up the equivalent of a cron job. Every day, or every week, your choice, the loop runs. It fetches failures, groups them, proposes a fix, and waits for you.

The human stays in the middle, always

This is the part that gets skipped in the excitement of automation. The loop is an accelerator. It is not a replacement for your judgment.

Eval changes need your sign-off before they run. Agent changes need your sign-off before they ship. The loop proposes. You approve. That is the design.

Aparna put it cleanly:

“Get data in, get an eval set up, give it criticism and let it go run on a loop.”

The criticism is still yours. The speed is Claude’s.

What compounds and what does not

The teams that build this loop are getting smarter with every cycle. Each round of failures produces better calibrated evals. Better evals produce more accurate agents. More accurate agents produce fewer failures. The signal to noise ratio improves over time.

The teams that skip the loop keep starting from the same baseline every sprint.

4. The new PM operating system

Step back from the mechanics for a second.

What you just learned is not a new feature to add to your existing workflow. It is a different operating system for how you do the job.

What the old operating system looked like

Monday morning. Open Linear. Read 40 new issues. Mentally rank them. Write a summary for standup. Two hours gone before you have typed a single Slack message.

Thursday. Pull up Gong. Watch three customer calls. Try to find the pattern. Write a themes doc that gets skimmed in a meeting and never opened again.

That cycle has a ceiling. There are only so many issues you can read. Only so many calls you can watch. Only so many sprints in a quarter.

What the new operating system looks like

The new operating system runs differently. Your agent already ran overnight. It pulled the latest issues, scored them by severity, recency, and reaction count, and wrote the PM report. It is sitting in your repo when you open your laptop.

Your job is a five-minute scan. Do you agree with the priorities? When you disagree, that disagreement is your next eval.

As Aparna said:

“If you are doing things the same way you were doing them last year, you have not caught up yet.”

That’s it for today. If you have any doubts, Aparna demoed the full process in the episode. Run the simple commands shared above. Watch Claude Code vibe it.

See you in the next episode.

Get Transcript

Where to find Aparna Dhinakaran

• LinkedIn

• Arize

• X

Related content

Podcasts:

1. AI Evals with Hamel Husain and Shreya Shankar

2. Evals are the new PRD with Ankur Goyal

3. AI PM Crash Course with Aman Khan

Newsletters:

1. AI Evals for PMs: Everything You Need to Know to Get Started in 2026

2. Your Complete AI PM Course & Career Roadmaps

3. AI PM’s Guide to LLM Judges

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

• Customer.io: Send smarter messages using your product data

• Amplitude: The market-leader in product analytics

• Bolt: Ship AI-powered products 10x faster

• Ariso: Your AI chief of staff for meetings, follow-ups, and coaching

• Product Faculty: Get $550 off their #1 AI PM Certification with code AAKASH550C7

Today’s episode

If you haven’t coded before, you’re getting pinched on all sides. Builder PMs coming from technical backgrounds. Layoffs targeting the managerial class. It’s tough out there.

Today’s episode is the antidote. We show you a step-by-step process to graduate from tools like Lovable to multi-agent systems in Claude Code.

I’ve brought in Andre Albuquerque, who has worked with over 4,000+ students at Builders Camp, runs five business with Claude Code and … has never been a developer!

Live on the episode, he builds a fully functional product from scratch. And he describes things in the most easy-to-understand way.

If you have been putting off Claude Code because it feels too technical or intimidating to set up, this episode is for you.

Apple Podcast

Spotify

• If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

• If you want access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

Newsletter deep dive

Thank you for having me in your inbox. Here is the complete guide to Claude Code for non-technical PMs:

1. Why non-technical PMs are getting left behind

2. The four levels of building

3. What changes when you start building

Subscribe now

1. Why Non-technical PMs are Getting Left Behind

Andre identified three major blockers holding non-technical PMs back. None of them is your fault, but they are absolutely your problem to solve.

Blocker 1 - You are kept away from building

Thanks to corporate bureaucracy, non-technical PMs have essentially become the human API between customers and engineering. The job becomes more about translating requirements than anything to do with the actual product.

Andre has seen this from the inside across more than 4,000 product people in 30 countries. The pattern is the same everywhere: smart, capable PMs who have been systematically kept away from the one thing that would make them most valuable - building. In his words:

They’re bureaucrats. They’re stuck in JIRA, they’re stuck in Linear, they’re stuck in PowerPoints, they’re not actually building, they’re not pushing code, they’re not adding features, they’re dependent on their technical teams to do that.

Contrast that with AI-native organizations, where everyone builds. Even the CEO writes code. The Shopify CEO’s GitHub shows green every single week. LinkedIn literally scrapped its associate product manager program and replaced it with an associate product builder program.

One thing that is becoming increasingly clear is that companies expect you not just to “manage” things but ship real value. That is the gap you’re competing against, and the only way to survive this is to actively build.

Blocker 2 - The false ownership and Latency tax

When you label one person as the absolute “owner” of the product, you accidentally give everyone else a free pass to stop caring.

As Andre put it:

“You end up with squads that are completely disempowered. And it starts with the fact that there is a product owner rather than a team feeling like owners of the product.”

When you cannot build, you are forced to protect your territory using artificial authority, endless meetings, and rigid documentation. This protective behavior creates what I call the Latency Tax - the massive loss of time and context every time an idea gets passed down.

Think about how a standard feature gets built today.

A customer shares a painful problem → You write a PRD → Design turns it into wireframes → Engineering reviews it and estimates the story points. Finally, weeks later, it gets scheduled into a sprint.

By the time an engineer actually sits down to write the code, the original insight has passed through three departments over two to four weeks. That delay is your latency tax. When a PM builds the prototype themselves, that collapses to hours.

This is the same argument I made in my product strategy keynote. When a PM can build the functional prototype themselves, the Latency Tax drops to almost zero. That’s because the person who understands the user’s problem is the same person building the solution.

Your build will not be perfect, but it will be a tangible MVP focused on solving the user problem. Instead of handing down a static text mandate, you hand over something real that design and engineering can actually interact with and improve upon.

Blocker 3 - The Wrong Starting Point

This blocker is the most fixable - yet sadly, it’s the one that almost every online tutorial gets completely wrong.

“Open Claude Code and start prompting“ is a lame starting point, to be honest. Most tutorials assume you are buzzing with excitement and already know exactly what you want to create. But the basic reality is: as a non-technical PM, you were never taught how to build. Your entire career has been focused on scoping, strategy, and managing timelines.

Because of that, when you finally sit down to build, your brain automatically goes to a massive project like a marketplace or a SaaS platform. You think you need to clone a massive app just to practice. That feels way too complicated, completely useless for your actual day job, and impossible to finish - so you don’t even start.

Even if you do force yourself to try, you immediately freeze on step one. I know exactly how frightening a blank terminal screen is when you’ve never touched a command line.

The solution is a better entry point. Andre showed a specific four-level path that dramatically increases the odds you actually stick with this.

2. The Four Levels of Building

This is the exact four-level stack Andre uses to build your technical confidence without forcing you to learn how to code from scratch:

1. Level 1: Lovable only

2. Level 2: Lovable + Claude Code + GitHub

3. Level 3: Claude Code + Vercel + Cursor

4. Level 4: Multi-Agent Infrastructure Stack

One quick tip: do not try to jump straight to Level 4. Start with Level 1. It’s more fun that way.

Level 1 - Building with Lovable

Lovable is the right starting point for one reason: it is way less scary than Claude Code.

You open it up. You describe what you want to build in plain English, and boom - something appears on your screen. All of that without you writing a single line of code.

Andre’s first Lovable project was just a basic tool to manage his family’s vacation home availability. He highly recommends building a personal project first because it keeps things fun. Since there’s zero business risk, you can explore freely.

One underrated step at this level: ask Lovable to critique your own prompt before it builds. Ask it what you are missing. It will respond the way a good senior engineer would - listing your exact blind spots. That exercise will teach you faster than any product management course ever could.

The moment you feel comfortable on Lovable, you will naturally want more. You will want to build something more complex, more controlled, and that is exactly when you move to Level 2.

Level 2 - Lovable as QA Infrastructure

You are ready to tackle Claude Code, but vibe coding still feels scary. So what do you do?

Andre solved this beautifully by connecting Lovable and Claude Code through the same GitHub repository. You can push code with all the flexibility Claude Code offers, but you can still see the visual evolution of your product on Lovable.

Three things to know before you start:

First, Claude Code is not just a code editor. You can have a real conversation with it just like any other LLM.

I did a full Claude Code walkthrough with Carl Vellotti that takes you from zero to confident. If you’re starting fresh, start there.

Second, Branch and merge sound technical, but the concept is simple: A branch is a copy of your project. You build on the copy. When you are happy, you combine it back with the main version.

Third, Even after Claude Code saves the changes, you need to click Publish in Lovable to make the app live for real users. Before you publish, you have a private preview link. Share it with your team, test it on your phone. Make sure it looks right first.

Now, here’s the exact setup:

One critical rule: This sync only works in one direction. You must start on Lovable first. If you build a project in Claude Code first and push it to GitHub, you cannot port that code back down into Lovable. Start on Lovable, then extend it in Claude Code.

Level 3: Claude Code + Vercel + Cursor

Level 3 is when you start wanting speed.

The moment you start playing with Claude Code, you’ll see yourself very quickly wanting to build multiple features at the same time. That means working with multiple branches, and Lovable no longer fits the bill here.

The three tools at this level each do one job:

1- Claude Code: where you write and build 2- GitHub: where the code lives 3- Vercel: the bridge between your GitHub and your users

You may add Cursor for a visual layer. Andre actually runs Claude Code directly inside the Cursor terminal rather than using the standalone desktop app for two highly practical reasons:

First, Cursor has a great free plan. If Claude Code ever breaks or gives you a confusing technical error, you can simply paste that error into Cursor, and it will help you debug it.

Second, Cursor syncs with GitHub automatically once you log in -meaning all your projects link up without manual configuration.

The visual layout in Cursor also massively helps non-technical builders. You can see yellow branch lines sitting outside the main purple line. Yellow is where you are building. Purple is what users see. When you merge, they combine.

Also, I want to give you a heads up. When you first open Vercel, it will look complicated. Do not worry about most of it. Just find the preview link. Click it. See if the feature looks right. That is all you need to do.

If you want a step-by-step walkthrough of these exact stacks - Lovable, Claude Code, Cursor - I covered them from an experimentation angle in my Vibe Experimentation guide.

Level 4: The Machine That Builds The Machine

Level 4 is agents. Think of it like hiring a personal team, except your team is made of AI agents. Each agent has one specific job. They work together, and you act as the director, telling them what to create.

Andre calls it the Team Claude Config. It has three parts: a Claude.md, a set of agents, and a set of skills.

Grab Andre's Claude Config

CLAUDE.md - the culture

CLAUDE.md is simply a text file that loads every single time you start a Claude Code session. Think of it like a list of core rules your AI team reads every morning before starting work.

Andre calls it “the values of your Claude.” It holds your rules, constraints, and operating principles. The most important rule Andre puts in his file is this: “For every single task, call the PM agent first.”

As you build, Claude.md should constantly evolve. When you catch yourself repeating the same instruction - “always use product context,” “create .md files,” “ask a clarifying question before starting” - add it to Claude.md. The next session starts with that rule already in place.

I have iterated on my own CLAUDE.md over 100 times. If you want a battle-tested version of what Andre is describing, grab it here.

The PM agent

The PM agent never does the work. Its only job is to receive the task and decide which agents handle which parts. Andre’s instruction inside the pm.md file: “Never do the work yourself because there’s going to be some agent better than you.”

The agents Andre built mirror his actual team:

Researcher - user research, competitive analysis, market context Discovery - structured problem framing before any solution work begins Designer - UX best practices, design system knowledge, visual decisions Engineer - code quality, architecture review, prevents bad patterns from entering the codebase Implementer - the one who actually writes the code

I strongly discourage you from copying someone else’s setup. Build a setup that mirrors your own team. How do they work? What do they care about? Write that down and turn it into agents.

Gabor Meyer, a PM at Google, took this to another level and has 21 agents. I broke down his full setup here.

The 50/50 Rule

Here’s the number that surprised me most in this conversation: AI-native teams spend 50% of their time improving the infrastructure and only 50% shipping features.

50% infrastructure. 50% features. That’s the split.

The logic is simple: the better the infrastructure, the faster and cleaner the shipping becomes. If you ignore this split, you end up shipping slop.

Slop is what happens when you let the AI do all the thinking for you. So it gives you the output, which looks functional, the UI looks super cool, but it doesn’t actually solve anything for the end user.

When something ships badly, your first instinct is to fix that one thing. Change the button. Redo the layout. Ask Claude Code to try again. That is the wrong way.

The right way is to ask: why did this happen? What rule is missing from CLAUDE.md? Which agent made the wrong call? Fix that. Because the next time you’re going to have a feature, you don’t want that to happen again. You improve the machine so that in the future it becomes way easier to just focus on the problem, ask your team, ask your agents, and ship the feature better.

Three Skills that Block Slop

Slop enters your product at two points: when you define the work, and when the code gets written.

Andre runs three specific checks before building any new feature:

1- A Jobs to Be Done (JTBD) skill - what is the user actually trying to accomplish? Write it out clearly

2- An Opportunity Solution Tree (OST) skill - based on Teresa Torres’s framework - what problem are you solving? What are all the ways you could solve it? Pick one before you start.

I had Teresa Torres herself on the podcast to break down how OSTs work in the AI era. If you are going to use her framework as a skill, hear it from her first. She covers how AI can speed up discovery, but warns that AI summaries can miss 20-40% of important details if you pull humans out of the loop.

3- A MoSCoW skill - sort your requirements into must-have, should-have, could-have, and won’t-have. Be honest about the won’ts.

3. What changes when you start building

There is a kind of knowledge that only comes from building yourself. When you have built with Claude Code, watched it succeed, watched it fail, watched it produce slop, and watched it produce something genuinely useful, you develop what Andre calls builder’s gut.

This intuition is what creates Technical Empathy.

The most friction-heavy relationship in tech is between a non-technical PM and a Senior Engineer. This tension usually comes from the PM asking “How long will this take?” A question that reveals a total lack of technical empathy.

Andre told me about a feature he scoped before he started building. He estimated it would take his team a day. After he tried building a similar feature himself in Claude Code, he realized the “simple” request actually required restructuring how the database handled user sessions. What felt like a one-day ask was actually a three-day architectural change. He never would have known that without building it himself.

That experience repeats in small ways every week once you start building. You stop over-scoping AI features because you know where agents break. You write better specs because you understand what the agent is working against. You stop asking “can we add one more thing?” because you’ve felt the cost of scope creep in your own Claude Code sessions. You know what a branch conflict feels like. You know that “simple UI change” sometimes means rearchitecting three components.

Andre described the shift in how engineers respond to him now. Before he could build, standups felt adversarial. He’d push for timelines, they’d push back with technical objections he couldn’t evaluate. Now when an engineer says “that’s going to touch three services,” Andre knows what that means because he’s hit that wall himself. The conversation moves from negotiation to problem-solving.

“Imagine a reality where every single person in the product squad can do this for the actual backlog.”

That intuition sharpens every product decision you make. Engineers stop seeing you as an authority they have to answer to and start seeing you as a collaborator who actually understands the tradeoffs they’re navigating.

That is not a distant future. That is what these four levels lead to.

Get Transcript

Where to find Andre Albuquerque

• LinkedIn

• Builders Camp

• Andre’s Newsletter

• Andre’s GitHub

Related content

If you’re starting from zero, watch the Carl Vellotti walkthrough first. If you’re already in Claude Code and want to set up agents, the Gabor Meyer episode (21 agents) is your next step. If you want the team-level version of what Andre described, start with Hannah Stulberg’s episode on Team OS at DoorDash.

Podcasts:

• Claude Code and agents with Gabor Meyer

• n8n, Claude Code, and OpenClaw with Mahesh Yadav

• Claude Code with Hannah Stulberg

Newsletters:

• How to Ship Your First Pull Request as a PM

• I spent the last week building you a Team OS in Claude Code

• How to Build Product Strategy in the Age of AI: Step-by-Step with Claude Code

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by Arize: Ship AI agents and features faster, with fewer regressions.

Today’s episode

If you’ve gotten an enterprise Claude subscription, you want to get the most out of it.

And the first question is: When do you use Claude Chat vs Cowork vs Code?

The second question is: How do you use each of them best?

Today’s episode answers both questions. I’ve brought back Pawel Huryn for a record fourth time. You guys loved his prior episodes on discovery, AI PM, and n8n.

Now, he’s back with a masterclass on the Claude ecosystem for PMs:

Apple Podcast

Spotify

• If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

• If you want access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

The roadmap from here

We’ve built the complete guide to building a self-improving AI PM operating system across Cowork, Claude Code, and Dispatch:

1. Why chat is a dead end for serious PM work

2. The Cowork power setup

3. What Claude Code unlocks that Cowork cannot

4. The self-improving knowledge system

5. The 24/7 PM workflow

Subscribe now

1. Why chat is a dead end for serious PM work

We’ve all used ChatGPT and Claude Chat to do work. But you shouldn’t be anymore.

Chat has three hard constraints that make it a dead end:

1. No continuity across devices or sessions

2. No access to your real files or tools

3. No systems that persist and improve

Each one is worth marinating on for a little bit.

Constraint 1 - No continuity

If you start a complex task on your desktop and then get pulled into a meeting, you cannot pick up that same session on your phone. You cannot resume it from a web browser.

Your only option is to copy the entire conversation into a new context and hope the agent reconstructs your state correctly.

Cowork and code do not have this limitation. You can build a multi-agent system with file access, tool connections, and persistent project context.

Constraint 2 - No file access

Chat cannot read your desktop. It cannot organize your invoices.

In the episode, Pawel dropped a folder of mixed invoices - PDFs, images, different months, duplicates with different filenames - into Cowork.

Then he plugged in a single prompt. From there, the agent built a step plan, extracted dates from PDFs, identified duplicates using hash functions, created month folders, move files, and verified. Four folders appeared in seconds. Duplicates removed. Images that were also invoices got sorted correctly even though the prompt only said “PDF invoices.”

Chat would have described how to organize invoices. Cowork organized them. On a real desktop. In a real file system.

Constraint 3 - No persistence

Chat has very little controllable memory across sessions.

You cannot build a system that learns from its mistakes, extracts patterns from your data, and applies those patterns to the next task automatically.

We’ll show you how to in Code and Cowork.

Putting it all together

So how should you be using the three surfaces? Here is Pawel’s setup:

• 70% Dispatch + Claude Code web sessions

• 25% Claude Code locally

• 5% chat

But you don’t need to get there right away. You can use cowork instead of code, too (as I’ll show in the next section). The real key point is you should keep chat to 5%, and use more powerful surfaces for everything.

2. The Cowork power setup

So if you shouldn’t be using Chat, what should you use? The next level up is Cowork. It’s a user-friendly skin on Code. Here’s how to set it up like a power user:

Step 1 - Skills + plugins

I wrote about this on Monday. Skills are the new prompts.

The agent reads only the name and description first. If the description matches the current task, it loads the full instructions. If it does not match, the instructions never enter your context window.

This is progressive disclosure. You can have dozens or hundreds of skills loaded. The agent consumes context only on the ones it needs right now. Here is what that looks like in practice.

Pawel’s PM skills marketplace has soared past 11,000 GitHub stars. Inside, there are numerous plugins. Each plugin bundles multiple skills.

Two plugins I’d highlight you install now:

1. Product discovery plugin - analyze feature requests, brainstorm ideas, plan experiments, create tracking metrics, identify and stress-test assumptions

2. Product strategy plugin - Ansoff matrix, pricing strategy, competitive analysis, product strategy canvas generation

Now here’s the caution:

The real ROI is in iteration. Here is the process I have found works:

1. Install a marketplace skill. Use it on a real task.

2. The output has problems. Give Claude the specific failure. Not “this is bad.” Something like “the acceptance criteria are missing edge cases around null values and the priority labels do not match our team’s P0-P3 scale.”

3. Tell Claude to read the conversation, identify the root cause of the failure, and rewrite the skill from first principles so it does not make that mistake again.

4. Test the updated skill on another real task. Find the next failure. Repeat.

After five or six iterations, the skill handles 99% of cases correctly. I gave you a full skill to improve skills automatically on Monday if you want to accelerate this cycle.

You cannot just sit and use some magic technique to get it right on the first try. Build, test, give feedback, iterate. That is the whole game.

Step 2 - MCP connectors for real tools

Cowork connects to your actual systems through MCP servers - and some are even more bundled into connectors. I covered building a PM OS from these connections with Mike Bal a few months ago.

You want to connect Gmail. Google Drive. Slack. Analytics tools. User research platforms. CRM. The more data flows in, the smarter every skill becomes.

Now here is how connectors work in practice for a PM.

In the episode, Pawel ran a live email demo on his inbox. One prompt:

The agent connected to Gmail, counted, and reported back. It even drafted replies.

Here’s our recommendation: start drafting your comms with this.

1. Configure the Gmail connector to draft-only mode. The agent proposes replies but cannot send.

2. Configure Slack the same way. Drafts appear with a “send” button you approve manually.

3. After every session, the system reviews your edits to its drafts and learns your voice. Next batch of drafts gets closer.

This is the AI-drafted email workflow I wrote about last year supercharged. The agent drafts. You approve. The agent learns from your edits. Your approval rate climbs over time.

Step 3 - Drive Strategy

AI really compounds when you don’t just use it to do work but to be a stratgic partner.

In the episode, Pawel loaded a product strategy skill via slash command (which I recommend - it ensures the right skill fires instead of relying on auto-detection).

The agent asked clarifying questions defined by the skill, then generated a full product strategy canvas as a PowerPoint deck.

Two things blew my mind. First, Claude is dramatically better at using PowerPoint than it was even two months ago. There is no excuse to walk into a stakeholder meeting with a bad presentation anymore.

Second, the output reflected the specific skill definitions around north star metrics and guardrails. A good skill is the difference between generic AI output and McKinsey-level output.

If you put these three steps together, you’re now a Cowork power user.

3. What Claude Code unlocks that Cowork cannot

At this point you are thinking - Cowork does everything I need. Why do I need Claude Code?

The answer is not about raw capability. If you load development plugins into Cowork, it can do similar things technically.

The answer is about interface, control, and systems that grow past 20 files:

The explorer view and multi-file systems

In Cowork, if you want to find a file the agent created, you use the little build in-browser.

This hits a wall fast. The moment your system grows to 50+ files - knowledge databases, skill files, templates, infographics, invoices, contracts, brand guides - you need the explorer panel. It shows folder structure. You expand directories. You click to open any file. You see what the agent has created and where it lives.

Any PM building a serious personal operating system will need this view. The file system IS the operating system.

Claude Code also gives you features Cowork lacks entirely:

1. Hooks - scripts that trigger before or after tool calls. Automate quality checks. Block requests that fail conditions.

2. Subagents - parallel workers that handle sub-tasks independently and report results back.

3. Local MCP servers - credentials scoped to specific projects. In Cowork, MCP connections are global across all sessions. In Claude Code, you define which connections each project can access. This matters for security.

4. Lifecycle control - what happens before a tool is called, what happens after, ability to block or retry.

If you want the foundations, I walked through the full Claude Code setup for PMs and the team OS pattern in separate guides.

HTML infographics and the component library

Every viral infographic from Product Compass - the Anthropic team calendar, the Claude Code pricing breakdown, the 74-release tracker - was built in Claude Code. Not Canva. Generated as HTML, iterated through natural language conversation, and exported as PNG.

The mechanism is more interesting than the output.

The system maintains a growing library of HTML components. When a new infographic performs well on social media, the agent analyzes it. It extracts the layout patterns, the visual components, the density characteristics. These get stored as reusable building blocks.

When a new infographic is requested, the agent assembles from the library. New layouts from proven components. Each new output that works feeds the library. The components compound.

The process works because of context engineering - feeding the agent the right information at the right time. Feed it winning infographics. It extracts patterns. Feed it your brand system. It applies constraints. Feed it new content. It generates within both.

Building your second brain for agents

Karpathy recently presented a system for building a personal wiki with LLMs. A knowledge base for humans. Agents organize your articles, attachments, notes into a browsable second brain.

The approach in the episode is the inverse. Build a second brain for your agents, not for yourself. You are the curator. You feed in articles, screenshots, infographics, competitor posts. The agent does not just store them. It decomposes them.

Feed it 10 high-performing LinkedIn posts from a specific creator. The agent extracts:

1. Hook patterns - what opening structures correlate with high engagement

2. Sound bites - phrases and sentence structures that get quoted and shared

3. Voice archetypes - tonal signatures that differentiate one creator from another

4. Engagement metrics - what format, length, and topic combinations drive reactions

It organizes this by platform. X in one domain. LinkedIn in another. Substack in another. Each domain accumulates its own knowledge. When you ask the agent to write for a specific platform, it loads only that platform’s rules.

The agent that wrote for X a month ago is measurably better at writing for X today. Because it has more data, more confirmed patterns, more rules to apply.

You are curating a knowledge base that makes every future session smarter than the last.

4. The self-improving knowledge system

Every PM building with Claude hits the same wall. Your CLAUDE.md grows. You keep adding instructions, examples, good output, bad output. Eventually it consumes half your context window on every prompt. A grammar check carries the same overhead as a strategy task.

This is the most important section of this newsletter. It is the difference between using Claude and building with Claude.

The CLAUDE.md router pattern

Do not put detailed instructions in your CLAUDE.md. Your CLAUDE.md has one job - explain the project structure and tell the agent where to find specific knowledge.

Here is what goes in CLAUDE.md.

1. Project description - what this project is about, in two or three sentences

2. File structure - what is in each folder so the agent does not scan the repo on every prompt

3. Who I am - basic context. Role. Audience. Goals

4. Knowledge system instructions - how to find domain-specific knowledge, how to route to the right file, how to update knowledge after tasks

5. Workflow references - pointers to workflow files for specific operations like fetching Twitter data, analyzing LinkedIn posts, or drafting Substack notes

Here is what does NOT go in CLAUDE.md.

1. Detailed writing style rules - put in a voice file

2. Lists of good and bad examples - put in a patterns file

3. Platform-specific instructions - put in platform-specific files

4. Historical data or metrics - put in a metrics file

The index file routes everything. X rules live in X knowledge files. LinkedIn rules live in LinkedIn files. Product strategy frameworks live in strategy files. The agent loads only the relevant domain for the current task.

I covered the foundations of this in my context engineering guide. The CLAUDE.md router is the implementation layer that makes context engineering work in practice.

Rules, hypotheses, and rejected patterns

This is where self-improvement actually happens. When the agent analyzes data - posts that performed, offers that converted, candidates that got hired, experiments that shipped - it does not just summarize what worked. It generates three types of knowledge:

1. Rules - patterns confirmed across enough data to apply by default. Example from the episode - “achievement-as-proof hooks outperform achievement-as-point hooks.” Confirmed across 46+ posts. The agent applies this automatically to every new hook it writes.

2. Hypotheses - patterns observed but not yet confirmed. The agent tracks these with evidence counts. A hypothesis can be promoted to a rule when evidence accumulates, or demoted to rejected when counter-evidence appears. Example - “emotional diversification correlates with higher average engagement.” Currently hypothesis, not yet rule.

3. Rejected - patterns that were considered but disproven by data. The agent keeps these to avoid re-testing dead ideas. Example - a specific hook format that was hypothesized to work but showed negative results across multiple tests.

The agent manages all three categories without being told what to look for. It discovers its own patterns. It updates its own confidence scores. It organizes by domain automatically.

In the episode, the knowledge database had entries the builder had never seen before. Hypotheses the agent generated independently from analyzing patterns across hundreds of posts. That is the compound effect.

The three-line prompt that makes any domain learn

You do not need to build the full system from day one. Here are the three instructions you paste into your CLAUDE.md. They work for any domain - testing, marketing, strategy, release notes, customer offers, interview prep, whatever:

That is the entire mechanism. Show the system 10 good examples and 2 bad examples. Ask it to create something new. It reviews the rules it has already extracted. It applies the confirmed ones. You give feedback on the output. It updates the knowledge. The next output is better.

The knowledge self-organizes by domain. Pricing rules stay in pricing. Testing rules stay in testing. Marketing rules stay in marketing. You do not need to manage the taxonomy. The agent builds it from the data.

If you do nothing else after reading this, do three things this week.

1. Add the three-line self-improving prompt to your CLAUDE.md.

2. Feed it 10 examples of good work in one domain you care about.

3. Ask it to do one task in that domain and see what rules it applied.

That alone will put you ahead of most PMs still briefing from scratch every session.

5. The 24/7 PM workflow

If you can build a system that works while you are away from your desk, why would you limit yourself to working only when you are sitting at your laptop?

The answer most PMs give is “I would not.” The problem is they do not know how to set it up. So let’s explain it.

Anthropic shipped four remote surfaces - web sessions, remote control, Dispatch, and Channels. Not all four matter equally. Here is what does.

Dispatch as your mobile command center

Dispatch is a single chat interface on your phone (and desktop) that can start multiple background tasks simultaneously. You type “create an infographic for this text.” While it works, you type “how many emails did I get in the last two hours.” While both run, you type “analyze the last 5 posts by competitor X.”

Each task delegates to a separate agent thread. You monitor progress. You provide feedback. You approve outputs. You dispatch the next task. All from your phone.

The description from the episode was specific - “my life works much better integrated with life now. I do not have to have blocks dedicated to work. I go for shopping, I go somewhere with my kid and I just dispatch tasks. I provide text feedback in the chat. I look at the results. I dispatch another task and continue what I was doing.”

This is not about working 24/7. It is about decoupling PM work from the desk.

Code web sessions for focused work

When you need the full explorer view, the file tree, the terminal - use code web sessions. These run on Anthropic’s servers, connected to your GitHub repo.

The critical insight - put your entire operating system into GitHub. CLAUDE.md. Skills. Knowledge files. Everything syncs. Code web sessions point at your repo. Now you can access the same system from any device. Your laptop does not need to be online. This is also the most secure setup because everything stays on Anthropic’s infrastructure.

The pattern - start something on your desktop. Continue it on your phone through Dispatch. Finish it from a web browser through code web sessions. One system across every device.

I walked through the setup in my Claude Code guide. The GitHub sync step is the one most PMs skip.

When to use which (and the n8n question)

Here is the decision framework:

1. Dispatch - mobile, multiple lightweight tasks in parallel, quick feedback loops

2. Code web sessions - focused work with full file explorer, any browser, laptop offline is fine

3. Cowork - desktop knowledge work with files, the daily driver for most PM tasks

4. Claude Code - complex multi-file systems, hooks, subagents, HTML generation

5. Chat - grammar checks and one-off questions. 5% of your time

But here is the question everyone asks. If Claude Code can automate everything, is n8n dead?

No. And the distinction matters.

Everything above is personal automation. Your judgment stays in the loop. You approve drafts. You review outputs. You provide feedback. The agent suggests. You decide. This is where Claude Code wins.

Production automation is different. Customer ticket responses. Onboarding flows. Compliance checks. Data pipelines. For these, you need guarantees that Claude Code cannot provide.

Claude Code workflows are text files an agent interprets. The agent can respect the instructions. It can also ignore them. You cannot tell an agent “if the API fails, retry three times” and guarantee it will. You cannot enforce “verify customer email exists before sending” as a hard rule. You cannot prevent one customer from seeing another customer’s data with a markdown file.

For production systems, you need n8n or a similar workflow engine where the logic executes as code, not as suggestions. Conditional branches. Retry mechanisms. Access controls. Deterministic execution paths.

• Personal automation - Claude Code. Text files. Agent interprets.

• Production automation - n8n. Code logic. Deterministic execution.

Knowing which one you need is what separates a PM who ships from a PM who demos.

Final Words

The PM role in 12 months will not disappear. But it will compress vertically. Super individual contributor PMs at the bottom. CPO and CEO at the top. The middle thins.

Most of your time will be orchestrating multiple agents, switching contexts, assessing outputs. The trivial parts - tickets, debugging, presentations, release notes - will be automated. The PMs who survive are P-shaped or broader. Marketing. Strategy. Technology. Product. Customers. And enough about every area to delegate the work and assess the output.

Get Transcript

Where to find Pawel Huryn

• LinkedIn

• Product Compass Newsletter

• PM Skills Marketplace on GitHub

Related content

Podcasts:

1. n8n Masterclass with Pawel Huryn

2. Claude Code PM OS with Carl Vellotti

3. Claude Code Team OS with Hannah Stullberg

Newsletters:

1. The complete Claude Cowork guide

2. How to use Claude Code like a pro

3. Build your PM operating system

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Maven - Get a $675 discount off Gabor’s course with my code

2. Amplitude - The market-leader in product analytics

3. Testkube - The leading test orchestration platform

4. Land PM Job - My 12-week AI PM + Job Search Course starts Monday!

5. Product Faculty - Get $550 off their #1 AI PM Certification with code AAKASH550C7

Today’s episode

Here’s the problem with most Claude Cost demos: they stop at the prototype.

Nobody shows what happens next. You try to add a second feature. The first one breaks. The styling reverts to default. The code is so tangled that you spend more time debugging than you saved by generating.

Gabor Mayer showed me what happens when you stop treating Claude Code like a magic prompt box and start treating it like a team.

He is a PM at Google. He has not written production code in 15 years. But over the past several months, he has been building real mobile apps using 21 specialized Claude Code agents. Not prototypes that live in a demo. Apps that are on the App Store.

In today’s episode, he walked through the entire workflow live and share all the resources free.

Apple Podcast

Spotify

If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

Do you want to become an AI PM? I’ve created a course for you. Starts next week.

Newsletter deep dive

Thank you for having me in your inbox. Here is the complete guide to building a full AI development team in Claude Code:

1. Why one-prompt vibe coding fails

2. The 21-agent team architecture

3. The spec-first workflow

4. From design to code without touching either

5. What changes when PMs actually build

Subscribe now

Save this. The full 10-step playbook on one page. Everything below is the why and how behind each step.

Share

1. Why one-prompt vibe coding fails

Every PM I know has built something with Bolt, Lovable, or Replit. The prototype looks great. It runs. It impresses people in a Slack message.

Then you try to ship it to real users. And you hit a wall.

Blocker 1 - Context compression silently destroys your spec

This is the failure mode that nobody talks about in tutorials. When you give one agent one massive prompt, the model compresses context. Details get dropped. Not randomly. Strategically. The model decides what is “important” and what is not.

In the episode, Gabor defined a complete color palette. Oranges, neutrals, specific accent tones. The agent received everything. The output used none of it. The layout was there. The structure was solid. But every color was a default.

The reason is straightforward. When the context window is full, visual styling details are lower priority than functional logic. So the model drops them. Silently. Without warning. Without an error message. You just get generic output and wonder what went wrong.

The fix is not better prompts. It is context engineering. Smaller, scoped tasks. Each agent gets only the context it needs for its specific job. The designer agent gets the brand guideline. The CTO agent gets the architecture spec. Neither gets the full 50-page document.

Blocker 2 - AI-generated code compiles but is not maintainable

A Reddit comment that hit home for Gabor -

“Vibe coding is just the rebranding of unmaintainable, low-quality source code.”

This is the real prototype-to-production gap. The code works today. You can demo it. You can push it to TestFlight. But the moment you touch it to add a feature, three other features break. No naming conventions. Circular references between modules. Zero comments explaining why anything was built the way it was.

The fix is a dedicated code quality agent. Gabor calls his the Spaghetti Agent. It runs after every sprint and checks naming conventions, circular references, comment coverage, and structural debt. When he ran it on his codebase for the first time, it caught issues he never would have found manually.

If you are building anything beyond a one-off demo, this agent is not optional. I covered similar quality patterns in my AI testing guide and my AI evals deep dive.

Blocker 3 - No dependency mapping means cascading failures

When you build without organizing work into sprints, agents try to build features that depend on code that does not exist yet. Front-end components reference API endpoints that have not been created. Database queries call tables that have not been defined.

The Atlassian MCP currently cannot create sprints directly in JIRA. That is a real limitation. Gabor uses tags as a workaround. He tags tickets as Sprint 1, Sprint 2, Sprint 3 and maps dependencies between them manually before starting the build. Without this step, the entire multi-agent workflow falls apart.

Every PM who has gone from prototype to production with AI agents has hit at least one of these blockers. The ones who shipped figured out the workarounds. The ones who quit assumed the tools were the problem.

Here is what the three blockers look like side by side, and what flips the moment you stop one-prompting and start running a team.

2. The 21-agent team architecture

You do not need 21 agents to start. Three will get you surprisingly far. But understanding the full architecture shows you where the complexity lives and which roles to add as your projects grow.

Here is the full roster: four clusters, 21 roles, and the markdown file pattern that makes them portable across every project you build next.

2a. The core agents every PM needs

The System Analyst is the linchpin. It breaks down product requirements into technical specifications. It asks clarifying questions one at a time. It documents decisions in Confluence. It creates tickets in JIRA. Without this agent, every other agent operates on incomplete context.

In the episode, the system analyst asked 14 clarifying questions before a single line of documentation was written. Vector DB choice. Usage limit mechanics. Conversation history handling. Search fallback strategy. API provider. Minimum iOS version. Screen count. Naming conventions. Each question one at a time so the answers stay deep.

The prompt pattern that makes this work -

“Please act like a good system analyst. Ask clarifying questions until you have a complete and comprehensive understanding. Ask questions one at a time. Do not start writing documentation until all questions are answered.”

Two critical instructions. “One at a time” prevents the agent from dumping 25 questions at once. “Do not start writing” stops it from jumping ahead before the spec is complete. Different LLMs have different tendencies. Some love to start coding instantly. You need to explicitly constrain them. This is the same principle behind the prompt engineering techniques that work across any AI tool.

The Spaghetti Agent handles code maintainability. Naming conventions. Circular references. Comment quality. Structural debt. Born from that Reddit comment. When Gabor ran it on his codebase for the first time, it caught problems he never knew existed.

The UX Flow Architect creates clickable prototypes using Figma’s built-in prototyping arrows. This is a small but important detail. The early versions of this agent placed visual drawn arrows between screens instead of using Figma’s actual prototyping connections. The prototype looked like it had navigation. But when you clicked play, nothing happened. It took months of iteration to fix.

Each agent has a specific Claude Code agent markdown file that defines its role, its constraints, and its interaction patterns. The setup mirrors how you would build a Claude Code Team OS for a human team.

2b. The real blockers nobody warns you about

The Figma MCP color problem. When you connect Claude Code to Figma through the MCP and pass it your full specification, the screens look structurally correct but the colors are wrong. Not slightly wrong. Completely wrong. The model compressed the context and dropped your entire visual identity. The fix is to pass the brand guideline as a separate, focused input to the Designer Agent. Never bundle it with the functional spec.

The Atlassian MCP sprint limitation. The MCP currently cannot create sprints directly in JIRA. Gabor uses tags as a workaround. Sprint 1, Sprint 2, Sprint 3. It works. But it means dependency mapping is a manual step in the system analyst prompt, not an automated feature.

The consumer app vs Claude Code gap. An agent role you set up in the Claude consumer app does not automatically transfer to Claude Code. You need to define agents separately in both environments. The system analyst in your consumer app conversation is a different instance from the system analyst in your Claude Code agent folder. Your AI PM stack needs to account for this separation.

The $200 Max plan economics. On the Max plan, a major build session uses roughly 10% of your monthly allocation. That means you get about 10 full build sessions per month. For a side project, that is plenty. For a production workflow with daily iterations, you need to be deliberate about when you run multi-agent sprints.

2c. Why reusable agents beat fresh setups

Every painful lesson, every edge case fix, every API workaround gets encoded into the agent markdown file. The next project starts from a position of strength. The Spaghetti Agent that took weeks to calibrate on project one is immediately useful on project two. The UX Flow Architect that took months to stop drawing fake arrows works correctly from day one on every subsequent project.

This is the compound interest of building with agents. The first project is slow. The second is faster. By the fifth, your agent team is genuinely effective. Gabor’s Maven course walks through the full setup at maven.com/gabor/productbuilder.

The 21 agents are not the point. The point is that every role on a software team can be replicated by a scoped, reusable AI agent. Start with three. Add roles when you hit friction.

3. The spec-first workflow

Most tutorials start with the terminal. Open Claude Code. Start prompting. Start coding.

That is backwards. The workflow that actually ships production apps starts in the consumer app. On your phone. Possibly while walking your dog. The process maps cleanly to the PM OS framework that works for any complex project.

3a. Define the system analyst role first

Before you describe your app, you ask the LLM to define what a good system analyst does. This creates a behavioral framework that the agent will follow for the rest of the conversation.

The prompt -

“What is the difference between a good system analyst and a bad system analyst in a software development team? Be as detailed as possible.”

The response gives you a blueprint. Requirement elicitation. Stakeholder management. Process modeling. Dependency documentation. You then instruct the agent to act like a good system analyst.

This is the same principle behind AI agents for PMs. Define the role explicitly before assigning the task. It works in Claude Cowork the same way it works in Claude Code.

3b. Dictate, do not type

This is where superwhisper changes the game. In the episode, the app specification was dictated in a single long monologue. Technology stack (Flutter + Firebase). Data storage rules (device-only, no server-side user data). API key security (Firebase Secret Manager, never exposed to front-end). Usage limits (20,000 word cumulative cap with escalating cooldowns). Tone of voice (friendly but firm, like a 20-year referee friend). Vector database configuration (Vertex AI embeddings for IIHF rulebook and Situation Book).

Typing that specification would have taken 30 minutes and produced half the detail. Dictating it took five minutes and captured every nuance. The longest dictation prompt in the history of this podcast.

Here is the actual prompt, the five-step workflow it kicks off, and the two-word constraint - “one at a time” - that stops the agent from face-planting.

The key rule - even if you ramble, even if you are not perfectly concise, the LLM will understand. You lose nothing by over-specifying. You lose everything by under-specifying. This applies whether you are building a prototype or shipping to production.

3c. Documentation before design

The system analyst creates the full Confluence documentation before any design or code begins. Product overview. Technical architecture. AI agent specification. Data flow diagrams. API endpoint mapping.

Without documentation, every agent operates on partial context. With documentation, every agent operates on the same source of truth. I covered this exact approach in my PRDs guide. The principle is identical whether your team is human or AI.

The boring part of building is the specification. The exciting part is watching agents create screens and write code. But if you skip the boring part, the exciting part produces garbage. The PMs who understand product strategy already know this.

4. From design to code without touching either

Once the specification is locked, the workflow shifts from the consumer app to three parallel tracks. This is where the 21-agent architecture pays off and where most of the real-world friction surfaces.

Three tracks - design, tickets, build - running in parallel into four sprints. 72 minutes from idea to App Store submission. Here is the map.

4a. Design through Figma Make and Claude Code

Start in Figma Make. Go to Spotted in Prod. Take screenshots of apps you admire. Feed those into Figma Make to create a brand guideline. Typography. Color palettes. CTA buttons. Error states. Transitions.

In the episode, two inspiration images produced a full brand guideline. One of them was a photo of a laptop cover. Figma Make derived custom colors from the image without manual hex entry.

Claude Code then used the Figma MCP to build actual screens in Figma based on that style guide. Five screens appeared in real time. Each one matching the brand guideline. The Chrome DevTool MCP lets Claude Code visually verify designs in a browser, catching visual bugs the Figma MCP alone cannot detect.

4b. Tickets with the full team review

The system analyst creates JIRA tickets. The entire agent team reviews every ticket before development starts. This is the step that separates production builds from demo builds. Same product launch discipline, different toolchain.

Designer agent verifies screenshots are attached. Test Architect ensures test coverage. Spaghetti Agent sets naming expectations. Product Council confirms data storage policies. CTO Agent validates architecture. This maps to the AI observability principles I wrote about previously.

4c. Sprint execution with the dependency mapping workaround

Tickets organized into sprints using tags (Atlassian MCP workaround). Dependencies mapped. Database setup in Sprint 1. API in Sprint 2. Front-end in Sprint 3. Integration in Sprint 4.

“Claude, start building. Go for Sprint 1. Once done, Sprint 2, then Sprint 3, and so on. If you have any questions, ask.”

Multiple agents work in parallel. The coding phase is the fastest part. On the $200 Max plan, roughly 10% per session.

Everything before the code is the hard part. Once those are right, the code practically writes itself. This is true whether you are shipping to production as a PM or managing an engineering team.

5. What PMs gain by building with agents

If agents can spec, design, code, and test, what is the PM actually doing?

Making product decisions. The tools just got absurdly faster.

Gain 1 - Firsthand understanding of agent behavior

When you interact with agents daily, you develop intuition for context window limits, hallucination patterns, and compression behaviors. That intuition directly improves your roadmap decisions. You stop over-scoping agent features because you know where agents break down. You stop under-investing in evals because you have seen what happens without them.

Gabor has not written production code in 15 years. But he now understands agent behavior better than most PMs who have only read about it. That understanding compounds across every product decision.

Gain 2 - A portfolio that proves competence

A working app on the App Store is undeniable proof. Password-protect a section showing the build process. Confluence docs. JIRA tickets. Agent architecture. That portfolio item says more than any certificate. It says you shipped.

Gain 3 - Iteration speed that compounds

The first build is the hard part. The UX Flow Architect alone took months. The Spaghetti Agent needed weeks of tuning.

But once v1 ships, everything accelerates. New features take a morning. The reusable agent files carry forward every lesson. The PM who has shipped one app can ship the next in a fraction of the time. Not because the tools are better. Because their agents are better.

Stack those three gains over a year and the gap between PMs who build and PMs who watch stops being a gap. It becomes a moat.

You do not need to know how to code. You need a willingness to understand how software works and the patience to specify before you build. If you want to get started, my Claude Code guide walks through the full setup.

Where to find Gabor Mayer

• Maven course - Go from PM to AI Builder

• LinkedIn

• X

Related content

Podcasts:

1. My Claude Code PM OS with Dave Killeen

2. Claude Code OS Layer with Carl Vellotti

3. How to Design like OpenAI and Figma with Ed Bayes and Gui Seiz

Newsletters:

1. The complete guide to Claude Code

2. AI agents for PMs

3. How to build AI products

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Maven - Get discounts off Mahesh's courses with my links

2. Amplitude - The market leader in product analytics

3. Jira Product Discovery - Prioritize what matters with confidence

4. NayaOne - Airgapped cloud-agnostic sandbox to validate AI tools faster

5. Product Faculty - Get $550 off their #1 AI PM Certification with my link

Today’s episode

LinkedIn just scrapped its APM program and replaced it with an Associate Product Builder track, and introduced a Full Stack Builder career ladder alongside it.

What does it even mean to be a “builder PM”?

Well, tools only get you so far. Learning Claude Code is helpful, but means nothing if you don’t have an understanding of the underlying first principles.

That’s today’s episode.

Mahesh Yadav created one of our most popular episodes, with over 35K views on YouTube, and now he’s back. Earlier, he taught you AI agents.

Today, he’s teaching you how to become a builder PM:

Apple Podcast

Spotify

If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

I’m giving a free talk on how to get interviews at the top AI PM companies on Thursday (April 23rd) @ 9:00AM PDT. Grab your seat.

Newsletter deep dive

Thank you for having me in your inbox. Here’s the complete guide to “becoming a builder PM.”

1. What is a “Builder PM”

2. The Builder PM Tool Stack

   • n8n

   • Claude Code

   • OpenClaw

3. Mastering AI Agents

   • The first principles of agents

   • How to build self-improving agents

4. The 10-week roadmap to Builder PM

Subscribe now

1. What is a Builder PM

Every PM has already used Claude Code or ChatGPT to get something done. But if that’s where it stops, you are not yet a builder PM.

There are two kinds of builder PM, and they’re different jobs.

Type one ships customer-facing product without engineering handoffs. Customer to 10 paying users, solo. This is what the LinkedIn APB program is training for and what lands you the comp trajectory Mahesh describes.

Type two builds internal agents that automate their own PM work. PRD reviewers, competitive intel, data dashboards. Same skills, different trajectory: stay in your current org, ship 3x more, get promoted faster.

Most of this guide teaches type two. Type one is the aspirational endpoint.

Why this title change happened now

The title change is a response to something that actually shifted in the last six months.

Six months ago, most autonomous agents broke within minutes. METR’s latest benchmarks show frontier models now sustaining multi-hour jobs, with Opus 4.7 running coding tasks for 3 to 6 hours in practice.

When agents can only run for 3 minutes, the PM’s job is to prompt them. When agents can run for 6 hours, the PM’s job is to design the system those agents run inside.

Designing that system is what builder PM actually means.

The compensation trajectory, in case you need the motivation

Mahesh’s comp trajectory shows you the top end of what’s possible. He went from $120K at Microsoft to $1.3-1.4M in his last Google role (Senior PM, AI), with a rough doubling every two years. A few of his AI PM friends at Nvidia are at $2-2.5M.

Levels.fyi still pegs median Senior PM comp at top tech around $350-450K. What Mahesh is describing is the premium the US tech market pays specifically for builder-skill AI PMs, where companies approach you and offer 30-40% over your current number. The $2M+ tier is a small group.

International markets pay differently - builder skills still move comp in EU, India, and APAC, just with different absolute numbers and different negotiation dynamics. And if you’re in mission-driven PM work where comp isn’t the yardstick, the same builder skills still let you ship 3x more impact with the same team - which is the whole point.

You probably won’t hit $1.4M next year. The premium exists, though, it’s real, and the gate is builder skills.

One honest caveat for managers reading this. If your PMs get 10x leveraged, the obvious question is whether you need fewer of them. The early signal from LinkedIn’s restructure is that the answer is smaller pods with broader scope, not layoffs - they’re keeping headcount and pushing each builder to own more surface area. That may or may not hold elsewhere. Worth naming, because every VP Product on this list is already thinking about it.

The failure mode

Here’s the trap: PMs read about Claude Code on Twitter, try it for a weekend, get a mediocre output, and conclude “this is overhyped.”

They skipped the first principles.

That’s the answer to Gothelf. He’s right that execution capacity was never the bottleneck. Judgment was. Judgment sharpens when you ship 12 prototypes a year instead of 1. The builder PM still talks to customers. They just ship without waiting.

That’s what the rest of this guide is going to give you.

When building is the wrong bet

Not every PM job benefits from this shift. A few honest cases where the 10-week investment doesn’t pay off:

• If you work in trust & safety, healthcare decisions, financial underwriting, or legal compliance, your job is to prevent errors, not to generate throughput. An agent that ships 12 prototypes a year doesn’t help you. An agent that proposes a new UI flow in a payments product can actively hurt you. Build evals skills instead.

• If your product lives inside a regulated perimeter and your company has a year-long procurement cycle for new tools, you’ll spend your 10 weeks waiting for IT approval. Spend the time on AI literacy - reading papers, running evals on existing outputs - rather than on personal tool setup.

• If you’re a founder with zero PMs under you, skip the “automate your PRD reviews” framing entirely. Your version is shipping real product to real customers using these same tools. The skills transfer. The PM-productivity examples don’t.

Builder PM is the right bet for most mid-career PMs at product-led companies with data portability. That’s a large slice of the market, not all of it.

2. The Builder PM Tool Stack

Three tools. Three different jobs. Using the wrong one for the wrong stage of your journey will waste weeks.

Pick based on where you are in the journey, ignoring whichever tool has the most Twitter hype this week.

n8n

n8n is a visual workflow builder. You drag components onto a canvas and connect them. That’s it.

Everyone who is deep in AI dismisses n8n as “too basic.” They are wrong. n8n is irreplaceable as a learning tool because you physically see every piece of the agent architecture as separate nodes.

In the episode, Mahesh demoed building a contract analyzer in n8n:

1. Email trigger hits when someone sends a contract

2. 2. Gmail node pulls the MSA (master service agreement)

3. Data loader converts the file to text

4. Text splitter chunks it into 1,000-character pieces with 200-character overlap

5. Embedding model converts chunks to vectors

6. Vector database stores them

7. AI agent reads the playbook and flags risks

8. Gmail node emails back the analysis

9. Eval workflow runs against ground truth

Every one of those is a visible node on a canvas. When something breaks, you see exactly where.

The limitation is real. n8n has no code mode, no version control, no test suites, and no path to production beyond a simple webhook. It stops you around 10 customers.

For most builder PMs, use n8n for 2-3 weeks, then move on. The exception is simple internal workflows with clear webhooks and no collaboration needs - those can live in n8n forever and that’s fine. The “graduate from n8n” rule is about your learning trajectory, not about n8n being bad at its job.

Claude Code

Claude Code is where you build the real thing.

The same tool works for a PM with zero coding experience building skills in English, and for a senior engineer building production services on top of that PM’s work.

Mahesh showed his PRD review setup in the episode. Here is what it does:

1. He drops a PRD into a folder

2. Claude Code reads his checklist (ruthlessly specific, encodes Amazon PR/FAQ format, includes AI-specific criteria like “is this differentiated from ChatGPT or is it a commodity AI wrapper”)

3. The agent reads the document and writes inline comments back into the .docx file, using a Python library to handle the XML under the hood

4. Comments are strategic, not surface level. Things like “What prevents Datadog or the Big Four from building this?” and “How do you handle misclassification?”

That is already useful. But here is where it gets serious.

Every 30 minutes, a second agent runs. It opens the folder of recent reviews, compares the AI’s output against the version Mahesh actually shipped, and logs the deltas to a learner.md file.

When the same correction appears 5 times across 5 days, the learner emails Mahesh: “I want to update your checklist. Here is the proposed update.”

He reviews, approves, and the checklist evolves.

Every day, the reviewer is a little better than the day before. That is the real moat. Not the initial skill. The learning loop that compounds from your judgment. We’ll go deeper on this in Section 3.

The other things to build in Claude Code:

• Competitive intelligence with subagents (3 subagents researching 3 competitors in parallel, rolling up into one report)

• Prototypes from PRDs (clone competitor screens, modify them into working products, ship to customers for feedback in days instead of months)

• Data dashboards generated from your actual production data

Use Claude Code as your daily driver for weeks 4-6 of the roadmap. By the end, you should have at least 2-3 skills running on your real work, with learner loops on top of each.

OpenClaw

OpenClaw is a pattern. The pattern matters because enterprise agents will probably work this way at scale. GCP and AWS are already shipping sandboxed-agent primitives, and the architecture maps directly.

Here is what OpenClaw adds that Claude Code doesn’t have:

Delegation through existing channels. You send the agent a WhatsApp message. It goes and does the work on a separate machine. When it is done, it sends the result back through WhatsApp. You are not sitting in a terminal watching it think.

Full machine sandboxing. Instead of granting file-by-file permissions, you install the agent on a dedicated Mac mini or a VM. It controls the entire compute environment. (Side note: Mac minis with 32GB+ are 10-18 weeks out right now, and several high-memory configs are unavailable entirely. CNN reported Apple Store employees calling them “OpenClaw machines.”)

Model agnosticism. Connect any model, including open source. You are not locked into Anthropic’s rate limits, which every heavy Claude Code user complains about daily.

For PMs at big companies asking the obvious question: Google is not going to let you install OpenClaw on your work laptop. That is true. But Google is going to offer the same pattern inside GCP. AWS is going to offer it inside their infrastructure. The architecture of sending a message to a sandboxed agent that reproduces your problem, tries a solution, and returns results - that is how enterprise agents are going to work at scale.

If you’re at a regulated company (finance, health, government, legal) or your data can’t leave the corporate perimeter, the pattern still applies but the venue changes. Your version is Claude Code or an internal agent platform running inside your SSO and VPN. NayaOne-style airgapped sandboxes exist for exactly this reason. The architecture is the same. The compliance envelope is different. Build what you can inside your perimeter and push your infra team for the pattern on the outside.

Learn the OpenClaw pattern now and you’ll recognize the shape of enterprise agent platforms when they arrive.

3. Mastering AI Agents

Tools without first principles is how PMs waste 3 weeks. Here are the first principles.

The first principles of agents

Every working agent has 4 components. Every disappointing agent is missing at least one. Before you debug a prompt, debug the architecture.

Component 1 - Intelligence (The brain)

The model is the intelligence layer. My current default: Opus 4.7 for agent orchestration and skill authoring, GPT-5.4 for fast single-turn tasks where latency matters. You can run either. The rest of the architecture matters more than this choice.

On its own, it is a genius with amnesia. Ask it about neural networks, it answers perfectly. Ask it what Trump said about Iran this week, and if you picked a cheap model, it tells you its knowledge cutoff is June 2024.

The intelligence layer alone gives you knowledge cutoffs, generic answers, and zero session memory. A model without the other three components is just a more expensive way to use Google.

Component 2 - Tools (The hands)

Tools let the agent take actions. Without them, it can think but it cannot do anything.

In the demo, Mahesh added Tavily (a search tool) to the agent. Same question about Iran. This time, the agent searched, found the answer, and responded with current information. One tool turned it from useless to useful.

The essential tools for PM agents are search APIs (Tavily, Perplexity) for current information, file processors to read contracts and PRDs, MCP servers to connect to Gmail, Slack, GitHub, and your CRM, and bash execution to run scripts. PMs who skip this layer wonder why their agent cannot do anything beyond trivia.

Component 3 - Memory (The recall)

Without memory, the agent forgets everything between messages.

The demo showed this failure instantly. The agent searched and answered the Iran question. Two messages later, Mahesh asked “what conflict am I talking about?” and the agent replied that it saw no previous mention of a conflict. Adding session memory fixed it in one click.

Memory has three layers: session memory for the current conversation, long-term memory for patterns across days and weeks, and structured memory for specific facts the agent needs to retrieve reliably. Every PM agent that feels “dumb” is almost certainly missing one of these.

Component 4 - Knowledge (Your data)

This is where your agent stops being a generic chatbot and starts being valuable.

Knowledge = your company-specific context. Contracts, playbooks, competitive intel, product specs, customer research. Without it, the agent answers from the internet. With it, the agent answers from your actual data.

The RAG pipeline is straightforward: upload your documents, chunk them into 1,000-character pieces with 200-character overlap, convert to vectors, store in a database, and query when asked a question.

In the demo, Mahesh asked about payment terms and tariff impacts on company contracts. Without knowledge, the agent gave generic legal advice from the internet. With the MSA uploaded, the agent answered from the actual contract clauses.

That difference is the entire value proposition of building your own agents vs. using a generic chatbot.

How to build self-improving agents

This is the section almost nobody is doing yet. Which is exactly why it matters.

You automated one task. Your PRD review agent works. Your competitive analysis runs every morning. Then you notice the agent makes the same mistake three days in a row. It misses the same section. It applies the wrong standard.

Static skills produce static output forever. Learning systems compound daily.

Here is the learner loop architecture Mahesh runs on his PRD reviewer:

Step 1 - Every review creates a job folder

Timestamped. Contains:

• Who ran the review

• The input document

• The AI’s output

• The version you actually shipped after your corrections

Step 2 - A scheduled subagent runs every 30 minutes

It opens recent job folders. Compares the AI’s output against your modified version.

Step 3 - Differences get logged to learner.md

What the AI got right. What it missed. What you added. What you deleted.

Step 4 - A threshold triggers a proposal

When the same correction appears 5 times over 5 days, the learner proposes a specific checklist update: an actual edited version of the checklist line.

Step 5 - You review and approve

The new checklist becomes the input for the next review cycle.

The critical design decision is that the learner never updates the checklist automatically. It accumulates evidence, it proposes, you approve. You stay in the loop. The system still compounds.

After a month, your review agent catches things it missed on day one.

After three months, your checklist has evolved from your original draft into something shaped by hundreds of real reviews and your actual judgment calls.

Inside a team, you’ll want one canonical version of each skill in a shared repo, with a light review process when someone proposes a merge. Otherwise you get skill drift, five competing PRD reviewers, and nobody knows which one is current.

Static skills are where most PMs stop. The learning systems that compound from your judgment are what separate the builder PM who gets promoted from the builder PM who watched a YouTube video.

4. The 10-week roadmap to Builder PM

You have a full-time job and maybe 5-8 hours a week to learn this. Here is the sequencing, tested with real PMs in real cohorts:

• Weeks 1-3, Foundations. n8n only. Build one agent with all 4 components, run one real evaluation, build one multi-agent system. By week 3 you can explain the 4 components to a junior PM.

• Weeks 4-6, Build. Move to Claude Code. Pick one real weekly task and automate it with a skill. Add a learner loop on top. Spin up your first subagent system (3 competitors in parallel works well).

• Weeks 7-9, Delegate. OpenClaw time. Pick one complete job (not a subtask) and delegate it via WhatsApp to an agent on a VM. Iterate on the skill until the output is good enough to forward to your team unchanged.

• Weeks 9-10, See the pattern. Read the AI tool market through this lens. Every new tool: is it solving for context, actions, or evals? Is it a variant of the agentic loop? Could you build it yourself?

The biggest failure mode is skipping Phase 1. PMs who jump straight to Claude Code hit a wall around week 3, have no mental model for debugging, and conclude “Claude Code is overhyped.” Do not be that PM.

That decomposition skill from Phase 4 is also what’s showing up in L5 and L6 AI PM interviews. Based on reports from candidates I’ve coached through Land a PM Job and notes from Blind, some loops now include live building exercises where you’re expected to open Claude Code and produce a working prototype.

Candidates who default to Figma mocks when the interviewer is looking for a working prototype have lost offers. Figma is still the right tool for design work. The signal the interviewer is looking for is whether you can ship.

For hiring managers

In 6 months every PM resume will claim builder skills. Two questions cut through it:

1. “Walk me through a skill you’ve built and how the checklist evolved.” Real builders have version history. Fakes have a one-liner.

2. “Show me the learner.md.” Real builders have a file of failures. Fakes don’t.

If a candidate says they’ve used Claude Code but can’t name the four components of an agent or explain when they’d choose n8n over Claude Code, they watched a YouTube video. That’s different from building.

[Bonus] Episode Summary

Ten weeks. Three phases of building. One phase of seeing.

At the end, you are a PM who builds AI systems that get better every day.

In 2026, that is the difference between being automated and being the one doing the automating.

Get Transcript

Where to find Mahesh Yadav

• LinkedIn

• Maven Course

Related content

Podcasts:

1. Claude Code Team OS with Carl Vellotti

2. OpenClaw + Claude Code with Naman Pandey

3. Claude Code OS with Dave Killeen

Newsletters:

1. The complete context engineering guide

2. How to use Claude Code like a pro

3. Practical AI agents for PMs

PS1. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

PS2. The third cohort of my LandPMJob Program is filling up. Apply now!

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Bolt: Ship AI-powered products 10x faster

2. Amplitude: The market-leader in product analytics

3. Pendo: The #1 software experience management platform

4. NayaOne: Airgapped cloud-agnostic sandbox

5. Product Faculty: Get $550 off their #1 AI PM Certification with my link

Today’s episode

Sometimes, I get access to the wildest guests on this podcast. Today, we get the awesome opportunity to look inside the design processes at OpenAI and Figma:

• Ed Bayes leads design on Codex at OpenAI

• And Gui Seiz leads all AI design at Figma

And they worked with me to put together a masterclass on how to design in the AI era.

If you want to design like the leading AI companies, this episode is for you: complete with screen shares and everything else you need to adopt the new AI design workflow.

Apple Podcast

Spotify

If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

I’m accepting applications for my third LandPMJob cohort. Join Me.

Newsletter deep dive

As a thank you for having me in your inbox, here is the complete guide to the new code-plus-canvas design workflow:

1. Why the linear design pipeline is dead

2. The code-canvas loop

   • Codex to Figma

   • Figma to Codex

3. When to use which tool

4. The 5-step adoption roadmap

5. Total football for product teams

Subscribe now

1. Why the linear design pipeline is dead

You know the process. Everyone does. It was unchanged for 15 years:

Paper sketches. Wireframes. High-fidelity mockups. Developer handoff. Engineering builds it. Design files bug tickets because the spacing is off by 4 pixels.

Every single stage in that pipeline existed for one reason: the next stage was too expensive to start with.

Constraint 1 - High fidelity required engineering

Building something interactive used to require an engineer. So designers front-loaded cheap artifacts. Wireframes exist because prototypes cost too much. Handoff specs exist because designers could not write code.

Codex changed this. A designer can now build a functional prototype in minutes. No engineer required. No sprint ticket. No two-week wait.

Constraint 2 - Design files and code were separate worlds

A designer worked in Figma. An engineer worked in VS Code. Getting them in sync required handoff documents. Redline specs. Manual transcription of padding values and border radii.

The Figma MCP changed this. Design files and codebases now read from each other directly. One click. Pixel-perfect. No translation layer.

Constraint 3 - Feedback required the finished product

You could not test interactions in a wireframe. You could not feel tap targets in a mockup. You had to wait until engineering built it. Then you filed bugs. Then engineering fixed them. Then you filed more bugs.

Now you test interactions in code from day one. Before a single engineer touches it. The feedback loop collapsed from weeks to minutes.

What this looks like in practice

1. Inside OpenAI. The Codex design team has been shipping this way since December. Something changed when the models hit a capability threshold. Content designers are submitting PRs. PMs bring working prototypes to design reviews.

2. Inside Figma. Designers are working directly in staging. Monetization designers who never wrote code before are building technically complex prototypes. The phrase I keep hearing is there are no more P2s. The cutoff between ship and backlog is arbitrary now.

I covered the foundations of this shift in my AI prototyping for PMs guide. The difference now is that the tools on both sides have converged.

The old pipeline was not a design process. It was a workaround for expensive tools. The tools got cheap. The pipeline died.

2. The code-canvas loop

The new workflow is code + canvas in a loop.

The most common question about this workflow. Will it be lossy? Will things break when you translate between tools?

Short answer: it is already remarkably high fidelity. And it only gets better with every model improvement.

Direction 1 - Codex to Figma

You have a React app running locally. You have been building a composer system in code. You need to feel how buttons morph. How permissions prompts expand. Where tap targets land. Static mockups cannot answer these questions.

Now you want to go deep on the visual layer. Get pixel-perfect on a component. Swap icons. Test type scales.

Open the Codex desktop app. Drag the pop-out window over your running app. Type this -

Import my homepage and composer permissions state into a new Figma file. @figma

Three things happen -

1. Reads your file system. Codex understands the React structure, components, CSS

2. Snapshots the running interface. The Figma MCP captures your live app state

3. Opens it in Figma. Click Open in Figma. You get a real, editable design file

The result is not a screenshot. Everything is responsive.

1. Every padding value matches your CSS exactly

2. Every border radius is one-to-one

3. Every shadow transfers without manual transcription

4. You can select specific component nodes instead of full screens

The newest MCP improvement. The use Figma tool now references your actual design library. Components map to your design tokens and local styles. You are not working on a facsimile. You are working on a representation built from your real design system.

The snapshot is not a copy. It is a living bridge between your code and your canvas.

Direction 2 - Figma to Codex

The reverse direction is just as fluid.

You have been iterating in Figma. Changed the model name in the picker. Adjusted a color. Refined spacing on a card.

Copy a link to the modified component. Paste it into Codex -

Update my code with the change I made here [Figma component link]

Codex reads the link through the MCP. Diffs against your local codebase. Generates the code change.

For non-coding designers this is the unlock. You are no longer blocked. Make changes in Figma. Your engineer pastes the link into Codex. The change propagates automatically.

One engineer recently demonstrated aligning Storybook to GitHub to Figma with AI running the loop and finding diffs between them. The maintenance overhead of keeping design systems in sync, the thing every team complains about, can now be automated.

Three foundations that make this work

1. Name components well. Good naming is table stakes for humans. Essential for agents

2. Align design tokens with CSS tokens. Border radius, spacing scale, color palette. One-to-one mapping

3. Use annotations. Read annotations into the MCP so intent travels alongside the visual output

The handoff is dead. The loop replaced it.

Where AI still falls short

AI isn’t still where it needs to be for lossless movement. These areas are limited:

1. Shader effects do not translate to a static canvas

2. Complex CSS transitions cannot be fully represented in Figma yet

3. Edge case decisions that only exist in code because an engineer solved something that never got recorded in the design file

4. Web-specific effects that Figma’s canvas does not yet support

Annotations help bridge some of these gaps. But the designer’s judgment is still critical for the last mile.

The good news. Every model improvement makes this better. OpenAI’s model 5.4 produced a material jump in quality. Internal designers say it is meaningfully better at working with the Figma MCP than anything before.

If you tried this six months ago and gave up, try again. The reliability has crossed into daily-use territory.

The tools have limits. But those limits are shrinking every month instead of staying fixed.

3. When to use which tool

The biggest mistake teams make with this workflow. Trying to do everything in code. Or everything in canvas.

Match the tool to the question you are trying to answer.

Mode 1 - Go wide on ideas

Use the canvas.

You can see the whole flow in front of you. Rearrange screens. Explore lateral directions. Print it out and put it on a wall and tear it down.

The canvas is still the gold standard for divergent exploration. No code tool matches the spatial freedom of dragging artboards around and seeing twenty concepts at once.

When to pick this -

1. You are exploring completely new interaction paradigms

2. You need multiplayer collaboration with the whole team

3. You want to rally people around a hero image

4. You are doing deep design system work with tokens and color tests

Mode 2 - Test interactions in code

Use Codex.

How does the button morph? How does the composer resize? Where are the tap targets? What happens at different breakpoints?

Static mocks cannot answer these questions. You need something running.

When to pick this -

1. You are testing responsive behavior across mobile, tablet, desktop

2. You need to feel how transitions work in real time

3. You want to stress-test an engineering concept by forking a branch

4. You are building a prototype that needs real data

Mode 3 - Ship the last mile

Use both.

Build interactions in code. Pop them into Figma through the MCP. Go deep on pixel-perfect details. Push changes back to code. Ship.

This is where the loop becomes the most powerful. The round-trip that used to take a sprint now takes ten minutes.

When to pick this -

1. You are polishing a nearly finished feature

2. You need to fix button animations, loading states, string changes

3. Engineering is waiting and you want to unblock them without filing a ticket

4. You want to submit a PR yourself

I covered the decision frameworks in my Codex PM guide. The underlying principle is simple.

The question is never which tool. It is what are you trying to learn right now.

4. The 5-step adoption roadmap

If you are at a traditional company, and your design team follows the linear pipeline, and your company has not have procured these tools yet, here is the path:

Step 1 - Just start

Download the Codex desktop app. You do not need your company’s permission. Build something for yourself.

Real examples from inside OpenAI -

1. A GTM team member built an entire iOS app with zero iOS experience

2. A comms team member designed an interactive drag-and-drop seating plan in HTML

3. A design lead and his wife are building a Japan trip planner for their honeymoon

None of these people are engineers. They downloaded the app and started.

If you do not know what to build, ask yourself one question. What would I build if I could build anything? Then build it.

Step 2 - Get the loop working once

Install the Figma MCP plugin from the Codex app. Open a project you already have.

The checklist -

1. Import one screen from code to Figma

2. Make one change in Figma

3. Push that change back to code

4. Verify it worked

Get the loop working once end to end. This proves the pipeline before you scale it.

The foundation matters. Name your components well. Align design tokens with CSS tokens. Clean up your Figma file structure. If your files are messy it is harder for both colleagues and agents to get up to speed.

Step 3 - Start with polish not process

Do not replace your existing workflow on day one. Use code for the polish stage first.

1. The button animation that never feels right

2. The loading state engineering skipped

3. The string change stuck in the sprint backlog for three weeks

4. The spacing tweak that would take 30 seconds but requires a whole ticket

This is the lowest-risk entry point. You are not changing how the team designs. You are adding a new capability where the cost of iteration was always highest.

Step 4 - Shift the starting point

Once comfortable, change where design begins.

Instead of paper sketches or low-fi Figma frames, open Codex and describe what you want. Show the team something real in the first meeting.

What changes -

1. Edge cases surface in the first conversation instead of the third sprint

2. Product strategy discussions start with working software instead of static decks

3. Your wireframe gives the team more dimensionality to the problem than any mockup

4. Feedback is immediate because the thing exists

Step 5 - Use AI as your tutor

You will hit walls. You will not understand why a React component renders the way it does.

Ask. The AI is an infinitely patient tutor that never clocks out.

Questions that build facility -

1. Start simple. Can you build this? If it does something, ask how does that work

2. Go deeper. I just inherited this system. Can you explain the data architecture

3. Find gaps. Are there redundant systems? Look through the entire codebase

4. Learn structure. What is the difference between a layout page and a normal page

The people who succeed in this era are not the ones who already know how to code. They are the ones curious enough to keep asking questions.

The roadmap is not learn to code. It is learn to be curious in a world where every question gets answered.

5. Total football for product teams

The natural question from above is: If designers can code and engineers can design and PMs can prototype, do we even need separate roles?

Yes. And understanding why is the most important takeaway.

The three questions that define each role

1. Engineers ask how do we build this well

2. Designers ask how should someone experience this

3. PMs ask why should we build this at all

A designer who ships code still self-identifies as a designer. They still care about being the voice of the user. They still think about craft and flow and emotional experience. The medium changed. The mandate did not.

The skill layer accelerates the blurring

Inside Figma, designers are writing skills that teach AI how to design well. PMs write skills for product decision-making. Anyone can use anyone else’s skill.

1. A designer uses a PM’s skill to work through a strategic decision

2. A PM uses a designer’s skill to prototype a flow

3. An engineer uses a design skill to ship polish to production

The skill layer means you are no longer limited to a third of the picture. You can lean into adjacent domains without switching careers.

Why roles survive despite tool convergence

The constraint was never the role. It was the amount of time it took to become proficient in tools outside your domain. If you wanted to learn code that gated you. If you wanted to learn design that gated you.

Now those gates are open. What remains is natural inclination. Judgment. Taste.

The mental model is total football. In 1970s Holland, every player could play every position. The goalkeeper could attack. The striker could defend. But each player still had a natural spike. The team was more dangerous because everyone could cover for each other.

The phrase inside OpenAI - prototypes, not PRDs. PMs bring working prototypes to design reviews. They ship PRs to stress-test ideas with engineers. The artifact that aligns teams is now running software, not static documents.

The bottleneck has moved. If developers have been accelerated 10x, designers have been accelerated maybe 1.5 to 2x. Design can become the bottleneck if you are not coding yourself.

The tools are the same across roles. The questions are different. And the questions are what define the role.

Get Transcript

Where to find Ed Bayes

• LinkedIn

• OpenAI

• X

Where to find Gui Seiz

• LinkedIn

• Figma

• X

Related content

Podcasts:

1. Xinran Ma - Design with AI

2. Crash Course in AI Product Design

3. Codex PM Guide with Carl Vellotti

Newsletters:

1. AI prototyping for PMs

2. The PM guide to Bolt

3. Codex PM guide

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Bolt: Ship AI-powered products 10x faster

2. Jira Product Discovery: Plan with purpose, ship with confidence

3. Kameleoon: Leading AI experimentation platform

4. Amplitude: The market-leader in product analytics

5. Product Faculty: Get $550 off their #1 AI PM Certification with my link

Today’s episode

If you want to run a truly AI-pilled product team, you need to get everyone on a Team OS.

When (PM @ DoorDash, Author of In The Weeds) first described this concept I had never heard of it, but after she showed it to me, I realized it’s absolutely genius.

We’ve done a lot of Claude Code episodes on this podcast, but this episodes is something genuinely new and impactful for PMs.

It’s the architecture and steps to building a Team Operating System so that you can scale your impact as a PM and product team:

• You checking in every single thing you do to a repo

• Team members checking it before checking with you

• Everyone building one shared, compounding knowledge base

If you are building a team that runs on AI, this is the episode to watch.

Apple Podcast

Spotify

If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

I’m starting the third cohort of my Land PM Job program soon. Join us.

Newsletter deep dive

As a thank you for having me in your inbox, here is the complete guide to building a Team OS with Claude Code:

1. The Team OS structure

2. Context management theory

3. Scaling analytics across functions

4. How to write 10x docs with planning

5. The learning flywheel

Subscribe now

1. The Team OS structure

As a PM, you are the human router. Every question goes through you. Every answer lives in your head or in a doc no one can find. That does not scale when one PM supports 20 people across five functions.

A Team OS fixes this. One shared GitHub repo. Every function checks in their work. Any coding agent traverses it. Everyone self-serves.

Grab Hannah's Template Repo

Component 1 - The root Claude MD

The root file loads every session. Keep it to three things -

1. Doc index. A map telling Claude where every type of information lives. Without it, Claude runs expensive explore agents searching your file system. With it, Claude navigates directly on the first try.

2. Team roster with handles. Every person, their role, their Slack ID, their GitHub handle. Enables natural language queries like “Slack Alex about the bug from today’s call.”

3. Key Slack channels. Channel names mapped to IDs and purposes. “Send this in the product channel” just works.

Don’t stuff too much into the root file. If it is longer than one page, you are burning context on information not needed in 80% of sessions.

I covered the foundations of building a PM operating system in my setup guide. The root Claude MD is the single most important file in that system.

Copy-pasteable template -

# Team OS

## Doc index
- product/       - PRDs, strategy, competitive research, customers
- analytics/     - Metrics, queries, schemas, dashboards, playbooks
- engineering/   - Bug investigations, RFCs, technical designs
- team/          - Onboarding, retros, team-level docs
- .claude/       - Shared agents, commands, skills

## Team
- Alex Chen (eng lead) - @alexc (Slack) / @achen (GitHub)
- Morgan Li (designer) - @morganl (Slack) / @mli (GitHub)
- Taylor Kim (analyst) - @taylork (Slack) / @tkim (GitHub)
- Jordan Wu (strategy) - @jordanw (Slack) / @jwu (GitHub)

## Channels
- #product-team    - Daily standups and feature updates
- #eng-team        - Engineering discussions and PRs
- #data-insights   - Weekly metrics and analysis reports

Component 2 - Nested doc indexes

Every major folder gets its own Claude MD. These are not content files. They are navigation maps.

In the episode, a query about customers consumed only 3% of the context window. Claude did not touch analytics. Did not read engineering docs. Navigated directly to the right files.

Without nested Claude MDs, Claude runs explore agents across your entire repo. Burns tokens. Takes longer. Leaves less thinking room.

Template for every folder -

# [Folder name]

## Doc index
- subfolder/file - What it contains and when to read it
- subfolder/file - What it contains and when to read it

## Key context
1-2 sentences needed in 80%+ of sessions in this folder

Component 3 - The folder architecture

A production Team OS has three top-level sections -

1. .claude/ folder. Shared agents, commands, skills. Customer call summary skill. PR creation command. Weekly synthesis automation. Everyone uses these.

2. Product development. Customers, competitive research, PRDs, strategy docs, launch emails, meeting summaries, analytics (metrics/queries/schemas), engineering (bugs/RFCs). Organized by function, then product area.

3. Team folder. Onboarding guides, retros, team-level docs.

The full directory tree -

team-os/
├── .claude/
│   ├── agents/
│   ├── commands/
│   └── skills/
│       └── customer-call-summary.md
├── product/
│   ├── CLAUDE.md
│   ├── customers/
│   │   ├── CLAUDE.md
│   │   ├── acme-corp/
│   │   │   ├── CLAUDE.md
│   │   │   ├── calls/
│   │   │   └── summaries/
│   │   └── forge-labs/
│   ├── competitive/
│   ├── prds/
│   ├── strategy/
│   │   ├── plans/
│   │   └── vision/
│   └── workflows/
├── analytics/
│   ├── CLAUDE.md
│   ├── billing/
│   │   ├── metrics.md
│   │   ├── queries/
│   │   └── schemas/
│   └── onboarding/
├── engineering/
│   ├── bugs/
│   └── rfcs/
└── team/
    ├── onboarding/
    └── retros/

The ownership model matters too. On the team in the episode, functional leads own their folders but the whole team agrees on structure. The data scientist owns analytics. Engineers own bugs and RFCs. The PM owns product context. Strategy partners own customer calls.

A non-technical strategy partner who had never opened GitHub two months ago now puts up PRs every day. This is not just for technical people.

Every level of nesting is a context-saving decision. The more precisely you organize, the less Claude has to read. That is the whole game.

2. Context management theory

You open Claude Code. You paste in a PRD. A competitive analysis. Three customer call transcripts. You have already consumed half your context window before you even ask a question.

No thinking room left. That is the trap.

The Team OS is built on the opposite principle. Load only what you need, when you need it.

The four pillars

1. Context. The information in a given LLM session. What the model can access right now.

2. Context window. How much it can hold. ~1M tokens today. Seven to eight novels. But the docs produced by a single team far exceed that.

3. Compaction. When the window fills, information gets compressed. You lose fidelity. Compressed summaries are much less useful than originals.

4. Thinking room. The gap between loaded info and window size. This is where the model reasons. More context loaded = less room to think.

I wrote about these foundations in my context engineering guide with an OpenAI product leader.

The token efficiency framework

Not all context is equal. A well-structured repo has three tiers -

1. Tier 1 - Always loaded. Root Claude MD. Team roster. Channel map. Under 500 tokens. Loaded every session.

2. Tier 2 - Loaded on query. Folder-level Claude MDs. Doc indexes. 200-500 tokens each. Only loaded when Claude navigates to that folder.

3. Tier 3 - Loaded on demand. Actual content files. PRDs, transcripts, SQL queries. Hundreds to thousands of tokens. Only loaded when specifically needed.

The magic is in the split. If you only need to know what a metric measures, Claude reads metrics.md and stops. Does not pull SQL queries. Does not pull table schemas.

Someone who burns hundreds of thousands of tokens and hits their usage limit in 30 minutes generally has unstructured, unoptimized context. The Team OS eliminates this.

Failure modes that burn context

1. Flat repo with no Claude MDs. Claude runs explore agents for every query. Burns thousands of tokens just navigating. I have seen teams hit usage limits in 30 minutes because of this.

2. Overstuffed root file. Everything dumped into one Claude MD. Loaded every session whether needed or not. Kills thinking room before you even start working.

3. Full transcripts instead of summaries. A one-hour customer call is 10,000+ tokens. A structured summary is 500. The repo points Claude to summaries first. Full transcripts only if the summary does not answer the question.

The skill that fixes the third failure mode - a shared customer call skill. Every team member summarizes calls in the same format. Same structure. Same fields. Cross-customer analysis becomes trivial.

Here is the pattern. You have ten people taking calls with twenty different customers. Without the skill, every summary looks different. With the skill, Claude can compare apples to apples across hundreds of calls in seconds.

Context management is not a nice-to-have. It is the difference between an AI that helps and an AI that hallucinates.

3. Scaling analytics across functions

Your engineer is on call at 2 AM. Something looks wrong with billing. They need to check a dashboard. They need the query for churn by segment. They need the table schema to validate the data.

You are asleep. Your analyst is asleep. The engineer is stuck.

This section pays for the entire Team OS investment.

Layer 1 - Metrics, queries, and schemas

Organize by product area, then by data type -

analytics/
├── billing/
│   ├── metrics.md          # Definitions only
│   ├── queries/
│   │   ├── generation_success_rate.sql
│   │   └── churn_by_segment.sql
│   └── schemas/
│       ├── billing_events.md
│       └── subscription_tables.md
├── onboarding/
│   ├── metrics.md
│   ├── queries/
│   └── schemas/
└── dashboards.md           # Links to all dashboards

The split is intentional -

1. What are our billing metrics? - Claude reads metrics.md only. Low context.

2. Show me the churn query. - Claude reads metrics + query file. Medium.

3. Full metric + query + schema. - All three files. Higher but still targeted.

4. Point Claude at the raw database. - Everything loaded. Maximum burn. Maximum hallucination risk.

The prompt that shows the power - “How do we calculate generation success rate? Show me the metric definition, the SQL query, and the table schema.” Claude navigates to the right product area. Pulls exactly three files. Gives you everything.

Hook this up to Snowflake MCP or another analytics tool. Claude does the actual analysis using verified approaches your analyst checked in.

Layer 2 - Playbooks and verified approaches

Your data scientist has a process for investigating funnel drop-off. That process belongs in the repo as a playbook.

When you ask “why are users dropping off during custom domain setup?” Claude finds the playbook. Follows the same methodology your analyst uses. Produces a result you can trust.

This is how you kill hallucinations in data analysis. Not by hoping. By giving Claude verified approaches.

The failure mode to watch for - not having the analyst audit the playbooks. If Claude follows a wrong methodology, it produces confidently wrong results. Your data scientist must own and verify the analytics folder.

Layer 3 - The feature launch gate

Make this non-negotiable. A feature is not launched until the repo is updated.

The checklist -

1. Metric definitions for the new feature checked in

2. SQL queries verified by your analyst

3. Table schemas documented

4. Dashboards linked

5. Playbooks for common investigations created

This is how shared context compounds. Every sprint, the repo gets more valuable. Every function gets more self-sufficient.

I covered the broader launch process in my product launch playbook. Add repo updates as a hard gate.

A second gate worth adding - the repo is part of the PRD itself. When you write a PRD for a new feature, include a section specifying what metrics, queries, and schemas will be checked into the repo as part of the launch.

The repo is not documentation. It is the system that makes everyone on the team faster every single day.

4. How to write 10x docs with planning

You type a prompt. Claude makes all the decisions. You get something back. You spend two hours fixing it.

That is not Claude Code’s fault. That is a planning failure.

The junior employee metaphor is perfect. You hired a brilliant junior. Then gave them zero guidance.

Three prompting tiers

1. Basic prompt. Type request. Claude decides everything. Unpredictable quality. Fine for quick lookups. Terrible for strategy docs.

2. Lightweight alignment. Add “give me a proposal first.” Claude proposes an approach. You correct direction in 30 seconds. I use this for anything with even slight ambiguity.

3. Full plan mode. Press Shift+Tab twice. Claude’s bias for action is removed. Cannot execute until you approve the plan. This is where real quality happens.

In the episode, even the lightweight alignment produced dramatically better results. Claude used the repo context to generate a competitive analysis proposal without being told to. Found the right folders. Read the right docs. Proposed the right structure.

The full planning process

Five phases for a strategy doc -

1. Phase 1 - Load context. Claude reads competitive research, vision docs, writing guides in parallel. Structured repo = only relevant files loaded.

2. Phase 2 - Ask user questions. Claude uses the ask-user-question tool. Who is the audience? What is the focus? Should we update competitive intel?

3. Phase 3 - Build the plan file. Section-by-section document structure. You read the plan. Push back. Refine.

4. Phase 4 - Push your thinking. The advanced move -

Use ask-user-question tool to push me on my thinking.
Help me consider other angles for this document.
Challenge my assumptions. Take as long as you need.

Claude interviews you. Catches gaps. Suggests sections you missed.

5. Phase 5 - Review agent prompts. For complex plans, check what each agent will be prompted with. What files will it read? What writing guide? Skills auto-invoke only ~70% of the time. Always explicitly specify in the plan.

Here is the thing. Most people rush in. Let it write a bad first draft. Then yell at it for getting things wrong. The fix is not better prompting after the fact. It is better planning before you start.

I covered the Claude Code setup foundations. Plan mode is where intermediate users become advanced.

Parallel agents and temp files

A single agent cannot read 40 context files and write a great long doc. Context fills. Quality degrades.

Fix these by doing followling-

1. Split the document across agents. Each gets specific context files and your writing guide.

2. Each agent writes to a temp file. If ten agents return work to the parent simultaneously, the context window overflows. You lose everything. You must prompt Claude to use temp files - it does not always do this automatically.

3. The orchestrating agent compiles the final document from temp files.

Two more advanced techniques -

1. Verification prompts. Tell Claude how to self-check. Require sources for claims. Use Playwright MCP to verify front-end output.

2. Save plan files to the repo. Native plan files get wiped every 24-72 hours. If you spent three hours on a plan, save it. Next time you start at 80% done. Your team can reuse it. OpenAI published that they made plan files first-class artifacts of their engineering repos.

The decision matrix for when to plan -

1. Straightforward task, clear scope - lightweight proposal. 30 seconds.

2. Complex doc, some ambiguity - full plan mode. 15-30 minutes on the plan.

3. Highly ambiguous, multi-phase research - full plan mode with thinking partnership. Could be hours spread across a day.

4. Recurring process - saved plan file in the repo. Start at 80% done every time.

Most people under-plan. That is why the output does not match what they wanted. The plan is not overhead. The plan is the work.

5. The learning flywheel

After 1,500 hours, Hannah is still iterating on her setup every single day. You should be, too.

The beginner’s mindset

Ask Claude to teach you. This one habit accelerates learning faster than anything else.

1. When you encounter a file you do not understand - explain why this is a YAML file and not markdown

2. When you download someone else’s skill - explain why this skill works and what could be improved

3. When you want to improve your repo - analyze this repository structure and suggest improvements

The prompt I run every day -

Research everything Anthropic shipped in the last 90 days. Read the top Claude Code influencers and their best posts. Compare my setup to the latest features. Tell me how to 10x my setup.

Claude’s training data gets stale. This prompt forces it to find what is new and apply it to your system.

I covered this progression in my AI PM learning roadmap. Claude Code fluency is one of the highest-leverage skills on it.

The automation-to-learning loop

If you only have two hours this weekend, ask one question - what can I automate to free up six hours next week?

The flywheel -

1. Automate one task → Free up time

2. Use freed time to learn → Improve your repo

3. Better repo → More automation possible

4. More automation → Even more time freed

Every MCP you hook up makes the system more powerful. The rule from the episode - any core software you use daily should be connected. The limit does not exist.

Shared automations compound the flywheel across the team. A weekly customer research synthesis that posts to Slack. A PR notification command that tags the right reviewer. A metrics check that runs every morning.

Mistakes that stall progress

1. Giving up after day one. Building a Team OS takes weeks. You are building infrastructure, not running a prompt. Commit to 30 days.

2. Copying without understanding. People download skills from the internet. Do not know why they work. When they break, they cannot fix them. Always ask Claude to explain first.

3. Treating it as a coding tool. The most misleading name in AI. Most PM time in Claude Code is writing docs, doing analysis, building prototypes, running automations. Not writing code.

4. Not clearing between tasks. Type clear when switching. Leftover context pollutes results.

5. Context rot. Not updating the repo. Claude uses outdated competitive intel. Outdated metrics. The repo must be a living system that every team member updates as part of their workflow.

The terminal is not scarier than a chatbot. Once you have typed into it for an hour or two, you feel comfortable. The barrier is psychological, not technical.

The PMs who build a Team OS this quarter multiply their leverage by 10x. The PMs who keep being the bottleneck for context just made themselves the slowest person on the team.

Get transcript

Where to find Hannah Stulberg

• LinkedIn

• In the Weeds Substack

• Her Github repo

Related content

Podcasts:

1. My Claude Code PM OS with Dave Killeen

2. Claude Code + Analytics with Frank Lee

3. Claude Code as PM OS with Carl Vellotti

Newsletters:

1. The ultimate guide to context engineering

2. Build your PM operating system

3. How to use Claude Code like a pro

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Bolt: Ship AI-powered products 10x faster

2. Amplitude: The market-leader in product analytics

3. Pendo: The #1 software experience management platform

4. NayaOne: Airgapped cloud-agnostic sandbox

5. Product Faculty: Get $550 off their #1 AI PM Certification with my link

Today’s episode

Claude Code hit $2.5 billion in annualized revenue in 12 months.

It is the fastest B2B software product ramp in history.

So why are most people still using it like a chatbot?

This is how most people use Claude Code. Type a prompt and get output. The context fills up. It compacts. You lose everything. You start over.

The top users flipped it. They built operating systems. Skills, sub-agents, file structures, hooks. Every prompt benefits from everything built before.

I sat down with Carl Vellotti for the third time. His first episode was the beginner course. His second episode was the advanced masterclass. Together they crossed over a million views across platforms.

Today is the operating system layer. If you are already an 80 out of 100 on Claude Code, this episode will bring you to a 95.

If you are living in Claude Code 8 to 10 hours a day and want to stop fighting the tool, this is the one episode to watch.

Apple Podcast

Spotify

If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

I’m putting on a free webinar on Behavioral and AI PM interviews. Join me.

Newsletter deep dive

As a thank you for having me in your inbox, here is the complete step-by-step guide to building your own Claude Code operating system.

Two starter repos you can fork today. Every file explained. Every technique from the episode plus what I have learned building my own PM OS.

1. Fork your starter OS

2. The CLAUDE.md file

3. The compounding file system

4. Mastering context

5. Skills that fix weaknesses

6. Data you can actually trust

7. The daily loop

1. Fork your starter OS

You do not need to build from scratch. Carl open-sourced his personal operating system at github.com/carlvellotti/carls-product-os. I open-sourced mine at github.com/aakashg/pm-claude-code-setup.

Carl’s repo comes with a BLANK-OS folder you rename and fill in, plus an EXAMPLE-OS with real content so you can see what a populated system looks like.

Step 1 - Fork and clone

# Carl's OS (lighter starting point, free)
git clone https://github.com/carlvellotti/carls-product-os.git
cp -r BLANK-OS/ my-workspace && cd my-workspace

# Or mine (41 skills, 7 sub-agents, $49)
git clone https://github.com/aakashg/pm-claude-code-setup.git my-pm-os
cd my-pm-os

Step 2 - Open in Claude Code

That is it. You are inside a pre-structured operating system. Every folder, every placeholder, every workflow is in place. Fill it with your data.

Carl draws a useful distinction. Workflows tell Claude how to create something. Templates tell Claude what the output looks like. Knowledge is reference material useful across projects. Project research is scoped to one project and gets archived with it. Your brand guidelines go in knowledge. Competitor research for a specific launch goes in the project folder.

You do not need everything on day one. Start with CLAUDE.md + GOALS.md. These two files give you 80% of the value. Add structure when friction appears.

2. The CLAUDE.md file

This is the most important file in your operating system. It is automatically loaded into context for every single message. Not just the first. Every one.

I have iterated on my CLAUDE.md over 100 times. What you think will work does not always. After lots of testing, here is what actually matters.

What goes in it

# About me
- Name: [Your name]
- Role: Senior PM at [Company]
- Team: Growth, reporting to [Manager name]
- Current quarter: Launching [feature], improving [metric]

# How I work
- Short, direct communication. No filler.
- Show reasoning before conclusions.
- When I say "spin up", delegate to a sub-agent.
- Check /knowledge/people/ before drafting any message.
- Use Jupyter notebooks for any data analysis.

# My tools
- GitHub CLI: installed (use for all git operations)
- Vercel CLI: installed (use for deployments)
- Firecrawl CLI: installed (use for web scraping)
- Tavily MCP: enabled (use for web search only)

# Current priorities
1. Ship auth redesign by March 28
2. Prepare board deck for Q2
3. Hire senior designer

# My skills
See /.claude/skills/ - auto-matching hook is active.

# Preferences
- Bold key phrases in documents
- Bullet points for 3+ items
- End sections with a punchy one-liner

The compounding trick

Update this file every week. Every time you hit an issue with Claude Code, pull the fix in. “Do not run research in the main session. Always use a sub-agent.” “When I say spin up, delegate to a separate instance.” Over months, this file becomes an incredibly precise instruction set. Claude feels like it already knows you.

I covered the CLAUDE.md deep dive in my Dave Killeen episode, where the CPO at Pendo told me his Claude Code system is better than the human EA he used to have. The CLAUDE.md was the foundation of everything he built.

If you only set up one file, make it this one.

3. The compounding file system

The fundamental difference between a Claude Code OS and just chatting with AI is that everything is a markdown file. And those files are alive.

In a normal AI conversation, everything lives in one chat window. Close it and it is gone. In a Claude Code OS, the AI reads and writes to persistent files. Every conversation makes the files better. Every file makes the next conversation better. That is the compounding loop.

The folder structure

my-workspace/
├── CLAUDE.md              ← Always in context, every message
├── GOALS.md               ← Quarterly priorities
├── .claude/
│   ├── skills/            ← Custom commands
│   └── hooks/             ← Auto-trigger scripts
├── knowledge/
│   ├── people/            ← Stakeholder dossiers
│   ├── company/           ← Strategy, positioning, competitors
│   └── research/          ← User research, industry reports
├── projects/
│   ├── [project-name]/    ← One folder per active project
│   └── _archive/          ← Completed projects
├── sub-agents/            ← Reviewer personas
├── workflows/             ← Process templates
├── templates/             ← Output templates (PRDs, OKRs)
├── data/                  ← CSVs, Jupyter notebooks
├── tasks/
│   ├── current.md
│   └── backlog.md
└── tools/                 ← Pre-built scripts

The knowledge folder

People files are where the real compounding happens. Create one for each stakeholder at knowledge/people/[name].md:

# Sarah Kim - Design Lead
- Reports to: VP Design
- Cares about: User research, accessibility, craft
- Communication style: Visual thinker. Wants mockups, not docs.
- Pet peeve: Decisions made without user data

## Recent context
- Pushing back on the auth redesign timeline
- Concerned about mobile accessibility
- Liked the competitive analysis I shared last week

## Meeting notes
### March 21 - Auth redesign sync
- Wants mobile prototypes before eng sprint
- Asked for usability report from last iteration
- Action: I owe her the report by Friday

Connect a meeting transcription tool like Granola via MCP. After every meeting, tell Claude: “Update the people file for Sarah based on today’s meeting.” Over time, when you draft a message to Sarah, Claude already knows she prefers visual communication and is concerned about mobile. The output is dramatically different from a generic draft.

Company files hold strategy, positioning, and competitive landscape. Research files hold user interviews, industry reports, and anything you reference across projects.

The projects folder

Every new task gets its own folder. When you come back the next day:

@projects/board-deck-q2/ Get up to speed on this project, 
then help me finish the revenue slide.

Claude reads every file in the folder. All the research, all the drafts, all the data. It picks up exactly where you left off.

Bootstrap fast. Do not build project folders one by one. Connect Linear or Notion via MCP and run:

Pull my last 5 active projects from Linear. For each one, 
create a folder in /projects/ with the PRD and latest status.

The people folder compounds across all projects. The project folder compounds within one. Together they mean Claude never starts from zero.

4. Mastering context

You have the file system. Now the biggest technical challenge: making sure Claude can actually use it without running out of context.

I covered the foundations in my context engineering guide. Here is how it works inside Claude Code specifically.

Step 1 - See your context

Run this right now:

/status line

Ask for a color-coded context meter. Green under 50%. Orange 50-80%. Red over 80%. Then run /context to see exactly what is eating it:

1. System prompt - ~2% (cannot change)

2. MCP servers - ~8%+ (each one adds permanent overhead)

3. Custom agents - ~4%

4. Conversation - grows with every message

Before you type a single message, 10% to 16% is already gone. Disable anything you are not actively using.

Step 2 - Delegate to sub-agents

Without sub-agents, a research task pushes context from 16% to 25%. Almost 10% of your window gone for one question.

With a sub-agent, the same task costs 0.5%. Claude spins up a separate instance. That instance does all the web searches and page reads. Your main session only gets the summary.

The numbers from the episode:

1. Without sub-agent - 16% to 25% context (+10%)

2. With sub-agent - 16% to 16.5% context (+0.5%)

3. Sub-agent used 10 tool calls and 30,000 tokens, all outside main session

Two ways to trigger this:

1. Inline - Add “spin up” or “use a sub-agent” to any prompt

2. Skill - Build a skill at .claude/skills/delegate-research.md that automatically delegates research tasks. Both repos include this.

Bonus trick: If Claude goes on a tangent, hit Escape twice. Roll back to before the bad prompt. Everything after it is erased from context completely.

Step 3 - Replace MCPs with CLIs

This is the hierarchy most people get backwards. MCPs eat context just by existing. CLIs sit on your machine with zero context overhead.

Connection Context cost When to use MCP High (always loaded) Last resort API Medium Custom integrations CLI Zero Always prefer

Andrej Karpathy confirmed it. The CLIs to install today:

1. GitHub CLI (gh) - Claude is aggressively competent at it. PRs, issues, repos, zero context cost.

2. Vercel CLI - Deploy, check logs, set env variables without leaving Claude Code.

3. Firecrawl CLI - Clean web scraping without MCP overhead.

The test: for every MCP you have enabled, ask Claude “does a CLI for this exist?” More often than you think, the answer is yes.

Every MCP you replace with a CLI is context you get back for actual work.

5. Skills that fix weaknesses

Every time someone says “Claude Code is bad at X,” the real problem is the same. Claude does not have the right tools or instructions. I covered the basics in my Claude Code guide. Here is the advanced version.

Tier 1 - Prompt-only skills

No code. No APIs. Just a well-written prompt. Anthropic has 56 official plugins you can install with /plugins. The front-end design one is the best example. All it does is tell Claude “do not look like AI.” Before-and-after quality is dramatic.

To build your own, create a file at .claude/skills/[name].md:

---
name: write-like-me
description: Use when the user asks to write, draft, or edit.
---

# Writing Style Skill

1. Short sentences. One idea per sentence.
2. Use "you" and "your" when teaching.
3. Open with the problem, not the solution.
4. Bold the key phrases.
5. End every section with a punchy one-liner.

Tier 2 - Tool-powered skills

Give Claude tools to check its own work. The make slides skill from the episode includes Puppeteer so Claude can screenshot its HTML output, measure text overflow, and fix issues automatically. Tell it “iterate 3 times before showing me” and the first result is already polished.

The web research skill replaces Claude’s unreliable default web search (basically Google first-page results) with Tavily for higher-quality search and Firecrawl for clean markdown scraping. Install Firecrawl as a CLI, not an MCP, to save context.

A key finding from a Google paper: literally pasting a prompt twice improves output. The practical version: after Claude executes any skill, tell it “double-check your output against the skill instructions.” Quality improves every time. This is the builder-validator pattern.

Whenever you catch yourself screenshotting and saying “fix this” repeatedly, that is a signal. Build a self-checking skill.

Tier 3 - Auto-invoking with hooks

Skills are supposed to auto-invoke from keywords. It does not work reliably. The fix is a user_prompt_submit hook that runs a lightweight script matching your message against skill keywords. Zero context cost. Instant. Claude still decides whether to use it.

This beats listing skills in CLAUDE.md because it adds no permanent context overhead and auto-updates when you add new skills. Both repos include the hook script ready to use.

The ask user questions tool

This was Carl’s favorite Claude Code feature, and one I had never used before this episode.

Tell Claude “use your ask user questions tool” and it generates a custom UI with checkboxes and input fields right inside the terminal. Instead of making assumptions, Claude asks you. Sometimes 10 questions. Sometimes 67.

Use it for:

1. Requirements gathering - “Use your ask user questions tool to grill me on all possible requirements for this feature”

2. Filling context gaps - “Ask me everything you need before starting this project”

3. Decision making - “Here are 10 research results. Help me decide which to implement”

The main reason people do not like AI output is assumptions. This tool eliminates them.

Skills are what you do. Hooks are how you compound. The ask user questions tool is how you eliminate assumptions.

6. Data you can actually trust

The biggest question PMs have about AI outputs: how can I trust this?

You are presenting numbers to your boss. They ask “where did this come from?” You need proof of work.

The answer is Jupyter notebooks. Tell Claude “analyze this data in a Jupyter notebook” and it creates a .ipynb file showing every query as a code cell, every result as output, every chart rendered inline. Renders natively in VS Code or Cursor.

The progression Carl demonstrated live:

1. Visualize - Load the CSV, see columns, shape, sample rows

2. Chart - Distribution charts, bar charts, segment breakdowns

3. Analyze - Correlation heatmaps, regression, statistical tests

At each step, your data scientist or manager can read every cell, verify the methodology, and check the results. The code is the proof.

The mental model: take the roles you work with and create versions of Claude Code for each one. The front-end design skill is the designer. Jupyter notebooks are the data analyst. The web research skill is the research assistant.

The difference between trusting AI and not trusting AI is visibility into the process.

7. The daily loop

This is where the operating system comes alive. Everything from sections 1 through 6 feeds into one daily workflow.

Create a standup skill that pulls from all your sources. GitHub CLI for recent commits. Tasks/current.md for priorities. Calendar for meetings. Linear for ticket status. Knowledge/people/ for meeting prep with stakeholder context already loaded.

One command. No tab switching. No manual assembly.

The compounding loop

You have a meeting with Sarah. Granola transcribes it. Your “update people” skill pulls key points into Sarah’s people file. Later, you draft a message to her about the timeline. Claude reads her file. It knows her concerns from the last meeting. The message is specific, not generic.

You finish a project. You move it to _archive/. Next quarter, you start something similar. You tell Claude “look at the archived auth-redesign project.” It reads the PRD, the user research, the launch results. Your new project starts with all the context of the old one.

You hit an issue with Claude Code. You add a rule to CLAUDE.md. Every future session inherits that rule. The same mistake never happens twice.

That is what an operating system is. Not a folder full of files. A system where every interaction makes the next one better. The compounding is the whole game.

How to start today

Fork the starter OS. Go to Carl’s product OS or mine. Clone it. Open it in Claude Code.

Fill in your CLAUDE.md and GOALS.md. These two files give you 80% of the value.

Set up the context status line. Run /status line. You will never unsee how fast your context fills up.

If you want the full system with 41 skills, 7 sub-agents, and the CLAUDE.md I have iterated 100+ times, get the PM OS here. If you want Carl’s free course that teaches Claude Code inside Claude Code, go to ccforpms.com.

Every person file you update, every project folder you populate, every skill you build makes the next prompt dramatically better. That is what turns Claude Code from a chatbot into an operating system.

Get Transcript

Where to Find Carl Vellotti

• Linkedin

• X (Twitter)

• Instagram

• Newsletter

• Claude Code for PMs course

Related content

Podcasts:

1. Claude Code Masterclass with Carl Vellotti (Ep 2)

2. Claude Code Initial Guide with Carl (Ep 1)

3. Claude Code PM OS with Dave Killeen

Newsletters:

1. Claude Cowork and Code setup guide

2. How to use Claude Code like a pro

3. My PM OS

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by

1. Product Faculty: Get $550 off their #1 AI PM Certification with my link

2. Amplitude: The market-leader in product analytics

3. Pendo: The #1 software experience management platform

4. NayaOne: Airgapped cloud-agnostic sandbox

5. Kameleoon: Leading AI experimentation platform

Today’s episode

There’s very few AI PMs who have seen inside a single company like Netflix, Meta, or Amazon. It’s even more rare to find one who has experience at all three.

That’s who I’ve brought in today.

Jyothi Nookula breaks out the whiteboard for a full deep dive covering all aspects of AI PM fundamentals and job searching for an AI PM job.

If you are trying to break into AI PM, this is the one episode to watch.

Apple Podcast

Spotify

If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

I’m putting on a free webinar on Behavioral and AI PM interviews. Join me.

Newsletter deep dive

As a thank you for having me in your inbox, here is the complete guide to becoming an AI product manager in 2026.

1. Taxonomy of AI PM Roles

2. The Core AI PM fundamentals

   • When to use AI

   • Which AI to Use When

   • The 3 Building Blocks of an AI Feature

3. How to Become an AI PM

Subscribe now

1. Taxonomy of AI PM Roles

Before you write a resume, update a portfolio, or prep for a single interview, you need to answer two questions.

What type of AI PM role are you targeting? And where in the stack do you want to sit?

Get these wrong and you’ll spend months preparing for interviews that test completely different skills than what you studied.

Axis One - Traditional PM with AI features vs AI Native PM

Type 1 - Traditional PM with AI features

This is 80% of what is labelled as AI PM jobs right now. Companies adding LLM capabilities to existing products.

1. A chatbot on a customer service portal

2. AI summarization inside a document tool

3. Smart search on an existing knowledge base

The core product existed before anyone bolted an LLM onto it.

Type 2 - AI native PM

The remaining 20%. Here the product IS AI.

1. OpenAI

2. Anthropic

3. Cursor

The product is fundamentally probabilistic. The value proposition is impossible without AI. You cannot build ChatGPT without an LLM.

There are 4x more open roles in the traditional category. If you are trying to break into AI PM, that is where the volume is. But the AI native roles are where the hardest product problems live.

Know which type you are applying for before you write a single line on your resume. The skills overlap. The interview questions do not.

Axis Two - Where in the AI Stack You Fit

After the role type, the second axis is where in the stack you sit.

The deeper you go, the harder the technical bar.

Layer 1 - Application PMs (60% of roles)

Own the end-to-end user experience. How users interact with AI. How to build trust. How to make AI reliable for everyday use.

This is the easiest entry point for someone converting from a traditional PM role. It builds on existing product management skills with AI knowledge layered on top.

Layer 2 - Platform PMs (30% of roles)

Build tools that other teams use.

1. Developer platforms

2. Model orchestration systems

3. Evaluation frameworks

4. Observability tools

You are not building for end users. You are building for other builders.

Layer 3 - Infra PMs (10% of roles)

Build the foundational systems that power everything above.

1. Vector databases

2. GPU orchestration

3. Model serving optimization

4. Kernel level compilation

The good news? The hardest roles are the smallest bucket.

The layer you choose determines the interview prep, the portfolio, and the companies you target. Pick your layer before you pick your first project.

2. The Core AI PM Fundamentals

Three things separate the AI PMs who ship from the ones who flounder: knowing when to use AI, knowing which AI technique fits the problem, and understanding the technical building blocks well enough to make real product decisions.

Let’s go through each.

2a. When to use AI

Here is where most teams get it wrong. Not a technical mistake. A product decision mistake.

MIT found that 19 out of 20 AI pilots fail. A key factor? Picking the wrong problems. Teams reach for AI when a rules-based system would have solved it faster, cheaper, and more reliably.

I covered the decision frameworks in my AI product strategy post. Here is the decision tree from the episode.

AI makes sense in three patterns

1. Pattern recognition in complex data. YouTube uses ML to identify patterns across billions of viewing sessions. The relationships are multi-dimensional. No rules could capture that.

2. Prediction from historical data. At Amazon, AI forecasts inventory needs based on hundreds of variables. Seasonal trends, promotions, weather patterns. Humans cannot process that many variables.

3. Personalization at scale. Individualised experiences for millions of users. Content recommendation engines are the classic example.

Heuristics win in four situations

1. Explainability is non-negotiable. AI models still have low explainability. Regulated industries need auditability.

2. Clear domain rules exist. Tax calculation. Tax codes are complex but explicit. Perfect for rules.

3. Data is limited. New features, new markets, no historical data. Do not force-fit AI.

4. Speed is critical. AI systems take longer to build. For MVPs, ship first. Add AI later.

The best AI PMs know when to say no to AI. That judgment is more valuable than knowing how to build a RAG system.

2b. Which AI to Use When

You decided AI is the right tool. Now which AI?

Most teams jump straight to LLMs when a simple ML model would have solved the problem in a week at a fraction of the cost. I wrote about this trap in my RAG vs fine tuning vs prompt engineering guide.

Traditional ML

Regression models. Random forests. XGBoost. Mature. Reliable. Still powers most of the AI you use daily.

Choose ML when:

1. You have structured data and need to predict or classify

2. You need the model to explain its decisions

3. Speed and cost matter

The PM question: Can I put this problem in a spreadsheet with clear input columns and an output I want to predict? If yes, start with ML.

Deep learning

Neural networks. Computer vision. Speech recognition.

Choose deep learning when:

1. You are dealing with images, video, or audio

2. Humans can do the task easily but you cannot write rules for it

3. Pattern recognition is too sophisticated for traditional ML

The face recognition example from the episode is perfect. You recognise a face instantly. Writing if-then statements to replicate that? Impossible.

Gen AI

LLMs. Diffusion models.

Choose Gen AI when:

1. Users need to interact conversationally

2. You are creating new text, images, or code

3. You need reasoning and synthesis across multiple sources

The optimization hierarchy (before fine tuning)

This is the most important takeaway for anyone building with AI right now.

1. Prompt optimization - better prompts, better results

2. Context engineering - what information gets loaded into the context window

3. RAG - dynamically retrieve relevant knowledge

4. Fine tuning - only if 1-3 do not work

80% of use cases get solved with RAG. Teams that jump to fine tuning because it is in the API documentation burn months unnecessarily.

These are not competitors. They are tools in your toolkit. The best AI products combine multiple approaches.

2c. The 3 Building Blocks of an AI Feature

These are three building blocks every AI PM must know. Each builds on the last.

Building Block 1 - Agents vs workflows

An AI agent makes decisions and takes actions on its own. You do not tell it the exact order. It understands the goal and reasons its way there.

A workflow is the opposite. Predetermined sequences. Every step defined. Deterministic outcomes.

The agent architecture

Four components.

1. Perception - how it receives input (text, images, APIs)

2. Reasoning - where models live (LLMs, classifiers, planning algorithms)

3. Execution - how it takes action (text generation, API calls)

4. Learning - feedback mechanism for improving over time

Building Block 2 - Prompts and Context Engineering

Prompts are the primary interface between you and the AI system. And most teams treat them as an afterthought.

Three layers to understand:

System prompts set overall behavior and personality. For a customer service agent, this means defining empathetic personality, professional tone, identity verification requirements, and escalation rules. The system prompt is the AI’s job description. Write a bad one and you’ll get a bad employee.

Few-shot examples are the most underrated technique in production AI. Show the model concrete examples of what a good response and a bad response look like. It sounds like extra work. In production, it is the single highest-leverage investment you can make. I’ve seen teams double response quality by adding 3-5 well-crafted examples instead of writing 3 more paragraphs of instructions.

Context engineering is where the real PM skill lives. I covered the full framework in my context engineering guide, co-written with an OpenAI product leader. The key insight: context engineering is a PM problem, not an engineering problem.

Three context layers matter:

1. Immediate context — the current conversation or task. What did the user just ask?

2. Session context — recent interactions and state. What happened earlier in this conversation?

3. Knowledge context — broader reference information. This is where RAG comes in.

Here’s why this matters commercially. Claude Sonnet has a 200K token context window. That sounds like a lot. It fills fast when you’re loading a knowledge base, conversation history, real-time data, and user prompts simultaneously. Every token costs money.

The teams that overspend on AI are almost always loading too much context into every interaction. A customer asks about their order status and the system loads the entire product catalog, the user’s full purchase history, and a 50-page return policy document. That’s wasteful. Context engineering is the art of knowing what to load and when — giving the model exactly enough information to answer well, and nothing more.

Building Block 3 - RAG (Retrieval-Augmented Generation)

RAG (Retrieval Augmented Generation) is the single most important technique for enterprise AI. I covered the fundamentals in my RAG vs fine tuning guide.

1. Take your company documents

2. Chunk them into smaller pieces

3. Convert chunks into vectors using an embedding model

4. Store vectors in a vector database

5. When a user asks a question, vectorize the query

6. Find the nearest matching chunks

7. Load those chunks alongside the user input into the LLM

8. The LLM generates a response grounded in your actual documents

The result: the LLM answers using your data, not its training data. It can cite sources. It stays current because you can update the document store without retraining anything.

Why RAG wins over fine tuning for most use cases

1. Works with your existing data immediately

2. No model retraining required

3. Update documents without touching the model

4. Cost is a fraction of fine tuning

5. 80% of enterprise use cases are solved here

The failure mode to watch for: bad chunking strategy. If your chunks are too large, the model gets flooded with irrelevant context. Too small, and it loses the thread. Getting the chunk size and overlap right is where most RAG implementations succeed or fail.

RAG is not glamorous. But it solves most of the problems teams try to throw fine tuning at. Start here.

3. How to Become an AI PM

You know the role taxonomy. You understand the technical fundamentals. Now how do you get the job?

Four steps.

Step 1 - Build products, not projects

Do not stop at a demo. Launch. Get real users. When things break, fix them.

“I built this product. It broke in these ways. Here are the challenges I overcame.” That beats any course certificate in an interview.

The difference between a project and a product: a project is something you show in a portfolio. A product is something someone else uses. Hiring managers can tell the difference in 30 seconds.

Step 2 - Three portfolio artifacts

You need three things in your portfolio, each demonstrating a different AI PM competency:

1. An app solving a real problem you have. Not a tutorial clone. Not a toy. Something you actually use. This proves you can identify a problem and ship a solution.

2. An agent that demonstrates goal-oriented reasoning. Show that you understand the difference between workflows and agents. Build something where the AI makes real decisions about which tools to use and when.

3. A RAG system grounding AI in domain knowledge. Pick a domain you know well. Build a system that answers questions using your own documents. This proves you understand the most commercially relevant AI technique.

Each with real users. Not static demos.

Step 3 - Get certified

The AWS AI Practitioner certificate pairs well with hands-on portfolio work. It signals to hiring managers that your knowledge isn’t just theoretical. It covers the fundamentals - ML concepts, model selection, responsible AI - in a structured way that complements your portfolio.

Certifications alone don’t get you hired. Certifications plus shipped products do.

Step 4 - Learn the PM cultures

Each company shapes a different PM.

1. Amazon - document-driven. PRFAQs and six-pagers reviewed up to VP/CEO level. PMs spend 40-50% of their time writing. You become an exceptional writer.

2. Meta - experimentation first. Ship variants. Test against control groups. Let data decide. The most sophisticated experimentation infrastructure in the industry. You live and breathe statistical significance.

3. Netflix - context over control. No rigid process. No documentation requirements. Trust and autonomy. You define your own swim lane.

Amazon teaches thinking backwards from customer pain. Meta teaches rapid testing. Netflix teaches the power of autonomy.

The companies you work at shape who you are as a PM. The insights compound. Get the experience at even one of these environments if you can.

Get Transcript

Where to find Jyothi Nookula

• LinkedIn

• NextGen Product Manager

Related content

Podcasts:

1. AI PM: Complete Course with Pawel Huryn

2. How to Become an AI PM with Marily Nika

3. AI PM Masterclass with Ankit Shukla

Newsletters:

1. The ultimate guide to context engineering

2. RAG vs fine tuning vs prompt engineering

3. AI foundations for PMs

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Kameleoon: Leading AI experimentation platform

2. Testkube: Leading test orchestration platform

3. Pendo: The #1 software experience management platform

4. Bolt: Ship AI-powered products 10x faster

5. Product Faculty: Get $550 off their #1 AI PM Certification with my link

Today’s episode

I have done four eval episodes now. We covered error analysis and LLM judges with Hamel and Shreya. Evals from first principles with Ankit. The PM’s role in depth.

But I kept hearing the same question: how do I actually build one from scratch?

That is what today’s episode delivers. We build an eval entirely from scratch. Live. On camera. No pre-written prompts, no pre-written data.

I sat down with Ankur Goyal, Founder and CEO of Braintrust, the eval platform behind Replit, Vercel, Airtable, Ramp, Zapier, and Notion. They just announced their Series B at an $800 million valuation.

Users are running 10x more evals than this time last year. People log more data per day now than they did in the entire first year the product existed. The episode explains why, and then we build one live.

We connect to Linear’s MCP server, generate test data, write a scoring function, and iterate until the score goes from 0 to 0.75. And along the way, Ankur drops what might be the most important reframe of the year: evals are the new PRD.

Plus, we cover the complete eval playbook for PMs:

Apple Podcast

Spotify

If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

If you want my coaching in your job search, apply to my cohort.

Newsletter deep dive

I took everything from the episode and combined it with lessons from four eval deep dives. Here is the complete practitioner’s playbook:

1. Why vibe checks stop scaling

2. The data-task-scores framework

3. Evals are the new PRD

4. Offline vs online evals

5. How to maintain eval culture

Subscribe now

1. Why vibe checks stop scaling

Open your AI product right now. Try three examples. Tweak the prompt. Ship it.

Two weeks later, support tickets start coming in about edge cases you never tested.

That is the vibe check trap.

When you do a vibe check, you are using your brain as a scoring function. You intuit whether the result is good or bad. That is an eval. It is just the version that does not scale.

Vibe checking is not wrong.

It is the earliest version of what eventually needs to become structured. In the error analysis process I covered in an earlier episode, the first step is reviewing 100 traces and taking notes. That is a structured vibe check.

Here is what matters: the best companies treat the transition from vibe checks to structured evals as a product decision, not an engineering one.

The wall hits when more people start using the product, more subject matter experts start contributing to quality, and more engineers start touching the prompts.

At that point, one person’s gut feeling cannot cover the surface area. You need software and process to execute at scale with predictable performance.

And here is the thing about durability. Models change every few months. Agent frameworks get replaced.

The prompt you wrote today might be irrelevant by summer. But if you invest in understanding what your users actually need and encode that as data, scores, and eval flows, that survives every model swap.

The companies that believe their agent wiring is their moat are highly likely to fail. The ones building true differentiation are the ones investing in evals.

The prompt is temporary. The eval is permanent. That is the whole game.

2. The data-task-scores framework

In the evals deep dive, we covered the full analyze-measure-improve lifecycle. That is the theory. Here is the practitioner’s shorthand: every eval, no matter how complex, is exactly three things.

Component 1 - Data

A set of inputs your AI product needs to handle.

During the episode, we generated test data for a Linear task management bot:

1. “What tasks are assigned to me?”

2. “Are there any overdue tasks?”

3. “How many tasks need to be triaged?”

You can optionally include ground truth answers. If you know the correct answer is 12, include it. If you do not, skip it.

Do not spend a month building a perfect golden data set before running your first eval. Auto-generate silly questions. Jump in. Start iterating.

This echoes what I covered on bootstrapping datasets with synthetic data. But in the live demo, we took it further. We used Opus to generate the data set in seconds, edited it by hand, and ran our first eval within minutes.

Component 2 - Task

A task takes an input and generates an output. It could be:

1. A single LLM call to GPT 5 Nano

2. A complex agent calling MCP servers, vector databases, and multiple models

3. Anything in between

At the end of the day, it produces some kind of output. That is what you evaluate.

During the demo, the first task attempt failed badly. The model just said “Happy to help with Linear” instead of answering. That zero score was expected and useful.

Here is what happened next.

We connected the Linear MCP server so the model actually had access to task data. Still failed. The model said “I am ready to help with Linear tasks” but never called a single tool.

So we changed the system prompt:

Don’t ask clarifying questions, just use the tools and figure it out.

This matters more than you’d expect. Models are post-trained to ask clarifying questions in conversational contexts. In an eval pipeline where you send one question and expect one answer, that post-training behavior works against you. You have to explicitly override it.

Then we loosened the scoring function. It was penalizing responses that referenced Braintrust tasks as sources, even though that was valid citation behavior. The scorer was too harsh for what we actually cared about.

Then we added few-shot examples to the prompt and specified which MCP tools to use.

Score went from 0 to 0.75 across the board. Each iteration touched a different part of the data-task-scores framework. That is the rhythm: identify which component is the bottleneck, fix it, rerun.

Component 3 - Scores

Scores take the input, the expected output if you have one, and the actual output. Their job: produce a number between 0 and 1.

That normalization is critical. It forces everything to be comparable. A week from now, when you run a new eval after swapping the model, you compare directly against today.

Do not overcomplicate scores. Use categorical options, not freeform numbers:

1. A = Full answer with citations

2. B = Partial answer

3. C = No answer at all

In the evals deep dive, I recommended binary Pass/Fail as the default. There is a case that not every score must be binary, but clear criteria beats freeform every time. The key is that you are not asking the LLM to generate an arbitrary number. You are giving it defined options and asking it to classify.

Data in, task runs, score comes out. Everything else is iteration.

3. Evals are the new PRD

This is the reframe that hit me hardest.

Think about a PRD from 2015. Unstructured document. A spec meant to communicate how to build something. The engineering team reads it, half follows it, the final product never matches.

The modern PRD is an eval. It is something that an engineering team who maybe does not know everything about the problem can use to quantify how well the software is solving it.

Step 1 - Encode your product intuition as data

Instead of writing paragraphs describing what the product should do, create a data set of inputs that represent what users actually need.

Step 2 - Use scores as success criteria

Instead of prose acceptance criteria, write scoring functions that quantify whether the software works.

An engineering team that does not know the domain can look at the eval and know exactly what good looks like.

When the eval passes and the product still feels wrong, that is on you. Your scoring function does not capture what matters. And that is actually a new area of leverage that PMs did not have before. You go from providing a qualitative spec nobody follows into something quantifiable.

I have been saying PMs should own error analysis for months now. This episode pushed that further: PMs should own the eval itself. The scoring function IS the modern PRD.

Step 3 - Close the distance gap

The farther you are from the end user, the more critical evals become.

Ankur framed this as a distance principle.

At Anthropic, the people training the models, building the harness, building the product, and using the product all sit inside one set of walls. Feedback circulates with almost no friction. In healthcare, engineers are not patients or doctors.

Ankur’s parents are both doctors, and he said when he talks to them about their work, he has almost no idea what they’re talking about. The jargon is specialized, the stakes are high, and the engineering team has no intuition for what matters.

Evals become the mechanism that bridges that gap.

Finance has the same problem with different jargon. Domain experts speak a language that engineers don’t, and patients become counterparties, but the structural issue is identical. PMs own the scoring function that translates between what users need and what engineering measures.

There was a Twitter controversy about Claude Code not using evals. This blew up enough that someone pinged me about it the day it happened because their boss was questioning whether evals even mattered.

Here is the reality: Claude Code is doing evals. People at Anthropic are using the product, providing feedback, and incorporating that feedback into iterations. That is a form of eval. They just don’t need formal process because the distance between builder and user is zero. That is not the case for 99% of AI products.

One more thing Ankur pointed out: a big use case for Braintrust has been helping companies collect evals they can share with labs so the labs can better support their use case.

When you have distance, you need a ledger to capture the information. Otherwise, how do you communicate it?

The best PRD in 2026 is not a document. It is a data set, a task function, and a scoring function the whole team can run.

4. Offline vs online evals

You have your eval running in a playground. Scores are improving.

That is only half the picture.

Workflow 1 - Offline evals test your hypothesis

This is what we built in the demo. A data set, a task, a scoring function. You run experiments on your laptop. You iterate on the prompt. You swap models. You see the score move from 0 to 0.75.

Fast. Cheap. Repeatable.

This is the AI equivalent of what used to require an A/B test. In the old world, non-deterministic problems meant expensive production experiments. Now you can run those experiments offline. That is why Braintrust customers average 12.8 experiments per day.

Workflow 2 - Online evals test reality

Take the same scoring functions and run them on real production logs. Every time a user asks a question, the scorer evaluates the response.

If your offline eval achieves 0.75 but your online eval consistently shows 0.3, something is not translating.

Workflow 3 - The flywheel

The real power is in the loop:

1. Online evals surface examples not performing well in production

2. Grab those examples, add them to your offline data set

3. Your offline evals now reflect real user behavior, not hypothetical test cases

4. Iterate until scores improve

5. Push changes, repeat

This is the same continuous improvement flywheel I covered in the evals deep dive. Now you can see what it looks like in practice at companies like Ramp, Notion, and Dropbox.

The best teams run this loop daily. More on the specific ritual in the eval culture section below.

Offline evals tell you what should work. Online evals tell you what actually does. The gap between them is your roadmap.

5. How to maintain eval culture

The biggest risk is not that your eval system is bad. It is that your team treats evals as a gate instead of an iterative loop.

Mistake 1 - Running evals only at the end

If you edit your prompt, test it on three examples, then run a full eval just to see if you can ship, you are doing it backwards.

Start from the eval. The best teams Ankur works with have a morning ritual. In standup, they pull up examples from the previous day’s production logs. They reconcile what they see with what their eval scores show. They find novel patterns that have emerged, add them to the data set, and grind on those failures all day. That is the job now.

Mistake 2 - Only having evals that pass

If every eval succeeds, you have blind spots. Either:

1. You do not understand what your users are hitting

2. You do not understand what is impossible today

Have failing evals. When a new model drops, rerun them first. Something interesting always happens.

And interesting does not always mean better. Gemini 3 Flash was outperforming Gemini 3 Pro on coding benchmarks but hallucinating more. Those nuances only surface with a full testing suite.

This is also how you plan product launches around model releases.

Braintrust shipped Loop this way. They built the eval for the feature before they shipped it. The eval failed on every model they tried. Then Claude 3.7 came out and there was a huge jump. The eval passed. They shipped.

Ankur said there was a watershed moment where 3.7 was the first model that could look at its own work and improve. Prior to that, he said models were like a dog looking at itself in the mirror. They didn’t recognize they were evaluating a virtual representation of themselves.

Create the eval for the feature you want. Watch the models. The moment a model hits the quality threshold, you ship.

Mistake 3 - Siloing evals to AI engineers

Evals should not be constrained to engineers. Product managers need access. Domain experts need access. The people who understand the end user should be improving the scoring functions.

Zapier was Braintrust’s first customer. Ankur told a story about their CTO Brian, who has been there for a long time and is very successful.

When Ankur met him, Brian introduced himself as a full-time AI engineer. This guy probably doesn’t have to work, but Ankur said he has never seen anyone nerd out about AI as much as Brian does. Companies like Zapier have pre-existing product-market fit.

If Ramp doesn’t work, it is very bad. They don’t have the leeway to screw things up. The quality bar is high, the scale is real, and vibe checks can’t cover the surface area. That is why these companies gravitate toward evals.

Braintrust removed user-based pricing specifically for this reason. They realized evals should not be locked behind engineering seat licenses.

Going back to the PRD analogy: if only engineers can see and edit the PRD, the product reflects what engineers think matters. Not what users need.

The teams that win are not the ones with the most sophisticated eval setup. They are the ones where evals are the first thing the PM opens every morning.

The whole game

Treat the eval as a first-class product artifact with a PM who owns the experience. The scoring function is the new PRD. The data set is the new user research. The flywheel is the new sprint cycle.

Open your platform of choice, write your first scorer, and run it on 5 real examples today. You will learn more in that first hour than in a month of vibe checking.

The prompt is temporary. The eval is permanent. That is the whole game.

Get transcript

Where to find Ankur Goyal

• X

• LinkedIn

• Braintrust

Related content

Newsletters:

1. AI Evals deep dive

2. AI observability for PMs

3. How to build AI products

Podcasts:

1. AI evals with Hamel Husain and Shreya Shankar

2. AI evals part 2 with Hamel and Shreya

3. AI Evals explained simply with Ankit

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify and YouTube.

Brought to you by:

1. Jira Product Discovery: Plan with purpose, ship with confidence

2. Vanta: Automate compliance, manage risk, and prove trust

3. Mobbin: Discover real-world design inspiration

4. The AI Evals Course for PMs & Engineers: You get $1250 with my link

5. Product Faculty: Get $550 off their #1 AI PM Certification with my link

Today’s Episode

OpenClaw is the hottest new AI tool that no one is helping you with. There’s zero real content on OpenClaw for PMs.

I covered what OpenClaw is and why it matters when it first went viral. But the rest of the web has been oddly quiet.

That is a big mistake. OpenClaw as software is as important as ChatGPT in 2023 or Claude Code in 2025.

It’s the software soon every PM will be using.

What I’ve found is that setup is the hardest part. So in today’s episode, I’m giving you a complete, step-by-step installation.

Plus, we cover five of the top PM use cases:

Apple Podcast

Spotify

If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

If you want my PM Operating System in Claude Code, click here.

Newsletter Deep Dive

As a thank you for having me in your inbox, here is the complete guide to setting up OpenClaw and building PM automations with it.

1. Why PMs should care about OpenClaw

2. How to Set Up OpenClaw

   • How to connect OpenClaw to Slack

   • The workspace file system

3. 5 of the Top PM Use Cases

   • Slack knowledge base

   • Automated stand-up summaries

   • Competitive intelligence on autopilot

   • Voice of customer reports

   • Smart bug routing by customer tier

4. Security and deployment

Subscribe now

1. Why PMs should care about OpenClaw

Go open Claude or ChatGPT right now. Ask it to check your Slack for blockers, scan your competitor’s pricing page, and post a summary to your team channel at 9 a.m. tomorrow.

It can’t do any of that.

Every LLM you use today is reactive. You ask, it answers. You close the tab, it dies. No access to your files, your tools, or your team’s communication.

OpenClaw is different in three ways.

It is proactive, not reactive. Set up a cron job and walk away. It executes at 3 a.m. when you are asleep. Scans channels, monitors websites, generates reports, posts them to Slack. No fingers lifted.

It is model agnostic. Not locked into one provider. Deep research? Plug in Claude Opus. Fast customer responses? Gemini Flash. Budget? Qwen 3.5 at 1/10th the cost. You control the model per use case.

It runs locally. Your data stays on your machine. No cloud lock-in. It reads and writes local files, which means you build a living repository of product docs, customer feedback, and competitive intelligence that grows over time.

Peter Steinberger described the difference between skills and tools perfectly. Tools are organs. Can the agent do it? Skills are textbooks. Does the agent know how to do it? OpenClaw has both.

The difference between OpenClaw and every other AI tool you use is simple. It acts. Everything else just answers.

Continue Reading Online

The three-command installation of OpenClaw

Most people who quit OpenClaw quit during setup. Not because it is hard. Because terminal commands feel unfamiliar.

The truth is, if you are not seeing red text, you are good. Yellow warnings are normal.

Two paths.

The one-click path (training wheels). Go to emergent.sh. They have a bot feature that pastes the install command for you. Replace your LLM key, hit enter, done. The limitation: no full control. You cannot use your own RAM, run a local model, or customize the workspace.

The terminal path (what most PMs should do). Three commands.

Step 1 - Install OpenClaw

Open your terminal. Paste:

npm install -g openclaw@latest

Finds the latest version. Installs it. If it fails, you probably do not have Node.js 22+ installed. Go to nodejs.org, install it, re-run.

Step 2 - Run the onboarding wizard

openclaw onboard

The wizard walks you through:

1. Accept the security warning (hit yes)

2. Choose Quick Start over Manual

3. Pick your LLM provider

4. Paste your API key

5. Choose your messaging channel

For the LLM provider, you have every option. Google, Anthropic, OpenAI, and more. If you want the best dollar-per-quality ratio, Gemini is strong. To create a key, Google “Gemini key create,” click Create, name it OpenClaw, copy, paste it into a note you won't lose.

After the provider, choose the specific model. Gemini 3 Pro for deep work. Any Flash model for speed.

Then choose a messaging channel. For PMs, pick Slack socket mode.

Step 3 - Hatch the bot

The wizard installs the gateway and hatches your bot. You see “Wake up, my friend.” You are live.

During onboarding, OpenClaw generates a soul.md file. This forces you to give your bot a name, a personality, interaction rules. Peter Steinberger designed it this way. He wanted a companion, not a disposable chatbot.

Skip skills and tools selection during setup. Easier to add later by editing the markdown files directly, or by asking the bot.

Three commands. 15 minutes. If you can install a Chrome extension, you can install OpenClaw.

How to connect OpenClaw to Slack

This is the highest-leverage setup. It puts your AI agent inside the tool you already use all day.

The terminal instructions during onboarding are bare bones. Here is the full walkthrough.

Step 1 - Create a Slack app

1. Go to api.slack.com/apps

2. Click Create an App

3. Choose From Scratch

4. Name it OpenClaw

5. Select your workspace

6. Click Create App

Step 2 - Enable Socket Mode

1. Left sidebar: Socket Mode

2. Toggle on

3. Token name: secret-token

4. Hit Generate

5. Copy and save the token

Step 3 - Add bot token scopes

Go to OAuth & Permissions. Scroll to Bot Token Scopes. Add:

1. chat:write

2. channels:history

3. channels:read

4. groups:history

5. groups:read

6. im:history

7. im:read

8. users:read

Each scope controls what your bot can see and do. Skip any your org restricts, but all matter for the use cases below.

Step 4 - Install to your workspace

Scroll to the top. Click Install to Workspace. Hit Allow. Copy the Bot User OAuth Token (starts with xoxb).

Step 5 - Paste tokens into OpenClaw

Back in your terminal, the wizard wants two things:

1. Bot Token (the xoxb one)

2. App Token (the secret-token from Socket Mode)

Paste both. Configure channel access. Allow all channels unless you have restrictions.

Once tokens are in, go to any Slack channel, invite your OpenClaw bot, mention it with @OpenClaw. It responds. You are running an AI agent inside Slack.

The workspace file system

Before building automations, understand where OpenClaw lives on your machine.

Everything is in a hidden folder at ~/.openclaw. On Mac, click Finder, right-click, Go to Folder, type a period, select .openclaw.

Inside the workspace:

1. soul.md - Bot’s personality and values

2. agents.md - Agent identity, safety rules, operational instructions

3. user.md - Your preferences (direct, no fluff, whatever you set during onboarding)

4. memory.md - Persistent memory that survives restarts

5. tools.md - Local configuration notes

6. heartbeat.md - Your cron jobs (starts empty)

All markdown files. Open in any text editor. Ask an LLM to generate content for them and paste in. Or ask OpenClaw itself to modify them.

There is also a gateway dashboard at 127.0.0.1:18789. Chat with your bot, view cron jobs, change API providers, check gateway status. Whatever you do in the dashboard reflects in the terminal and vice versa. Same brain.

If you want to swap from Gemini to Anthropic, go to Config > Secrets in the dashboard, change the model, save. Or just tell the bot in chat and it reprograms itself.

Terminal is your command center. Gateway dashboard is your control room. Both talk to the same agent.

3. 5 of the Top Use Cases for PMs

Setup done. Now let's put it to work.

The five use cases below are the ones Naman and I spent weeks building and testing on camera. Each one replaces a manual workflow you’re probably doing right now:

1. A Slack knowledge base that answers your team’s product questions from your own docs

2. Automated stand-up summaries posted before your first meeting

3. A competitive intelligence pipeline that runs while you sleep

4. Voice of customer reports pulled from every source into one place

5. Smart bug routing that triages by customer tier without you touching it

Plus the full security and deployment guide so you don’t end up WhatsApp-ing your mom a pairing code.

(paid subscribers only)

Read more

Check out the conversation on Apple, Spotify and YouTube.

Brought to you by

1. Pendo: The #1 software experience management platform

2. Jira Product Discovery: Plan with purpose, ship with confidence

3. Amplitude: The market-leader in product analytics

4. NayaOne: Airgapped cloud-agnostic sandbox

5. Product Faculty: Get $550 off their #1 AI PM Certification with my link

Today’s Episode

This episode is a real life example of how a CPO actually uses Claude Code to do everything:

• His morning daily plan

• His career planning

• His meeting prep

All with real data plugged into his real systems: CRM, calendar, etc..

He thinks it’s better than the human EA (executive assistant) he used to have. And in today’s episode, he demonstrates everything.

Dave Killeen has a 25 year career in PM, so when he embraces a new AI tool this thoroughly, it’s worth checking out:

Apple Podcast

Spotify

If your product team doesn’t have access yet, send this to your manager or skip-level.

If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

If you want my PM Operating System in Claude Code, click here.

Newsletter Deep Dive

As a thank you for having me in your inbox, here is the complete guide to running your entire work life in Claude Code:

1. The daily plan command

2. How to connect everything with MCP servers

3. Skills vs MCP vs hooks

4. The compounding file system

5. From backlog to PRD to Kanban board

6. Career planning inside your OS

7. How to get started today

Subscribe now

1. The Daily Plan Command

Here is the core problem with how most PMs start their day.

You open six or seven tabs. Calendar in one. CRM in another. Meeting notes buried somewhere. LinkedIn messages you meant to respond to three days ago. By the time you've assembled the picture, you've burned 30 minutes and lost your flow state.

The fix: one slash command. Five minutes later, you have a full daily plan.

Here is what the command does under the hood:

Step 1 - Check what intel has already run

Before assembling the plan, the system checks whether all of its intelligence digests have already run for the day. Has it pulled LinkedIn data? Twitter data? Newsletter summaries? If any are missing, it executes those first.

Step 2 - Pull structured data through MCP

It pulls your calendar, weekly priorities, quarterly goals, tasks, and CRM data through MCP servers. Your specific connections will vary. Dave connects to Clari (Pendo’s sales tool), Granola (meeting notes), LinkedIn via Phantom Buster, YouTube transcripts, and 120 newsletters. You start with whatever tools you already use. Calendar is the easiest first connection.

Step 3 - Assemble and surface what matters

Everything gets assembled into a single markdown page. Three priorities for the day. Which accounts need attention, because it has been listening to customer conversations through Granola. Which LinkedIn messages are connected to existing enterprise accounts. It even writes the Slack messages to send to his team.

One command. No tab switching. No manual assembly. The AI does it all for you.

2. How to connect everything with MCP, APIs, or CLIs

The daily plan works because every tool is connected.

Most PMs hear “MCP” and “API” and assume it requires engineering work. It does not! The process is simpler than most people expect.

Step 1 - Find the documentation

Go to whatever tool you want to connect. Your calendar is the easiest starting point. Find the API, MCP, or CLI docs.

Step 2 - Tell Claude to build the server

Tell Claude through voice:

Here is the API/CLI/MCP documentation for this tool. <I have an API key for you.> Connect with it.

Claude reads the documentation, builds the server, and you are connected.

Step 3 - Point the AI at new use cases

Once the server is built, you can ask Claude:

Knowing what you know of the API documentation, how can we make this even better? What could we do?

It comes back with use cases you had not thought of, because it has ingested all the API documentation into your system.

Dave’s rule is simple. If he has access to an API, he connects it. Everything and anything gets connected. The more data flows in, the smarter every command becomes.

Why MCP instead of raw API calls? Because MCP acts as guardrails. It defines exactly how the AI should interact with each service, what data to pull, and in what format. That makes the results far more deterministic than pointing Claude at a raw API and hoping for the best.

3. Skills vs MCP vs hooks

This is where most people building Claude Code workflows get confused. There are three building blocks, and each one does something fundamentally different.

Building block 1 - Skills (now synonymous with commands)

A skill is a plain English instruction file that tells the AI what to do when you invoke it. Think of it as a job description. “When I say /daily-plan, pull my calendar, check my goals, scan my intel digests, and assemble a plan in this format.”

Skills are flexible and easy to create. Describe what you want in a conversation and Claude writes the skill file for you. Dave has about 60 in his system. You’ll probably start with five or six and grow from there.

If you want something to behave loosely and benefit from AI judgment, make it a skill.

Building block 2 - MCP servers

MCP servers are structured integrations that tell the AI how to interact with external services. Tighter than skills. They enforce guardrails around what data gets pulled, what format it comes in, and what steps the AI follows.

Example: a task MCP server ensures tasks get created in a consistent way every time, always attached to the right project and the right strategic pillar. Without the guardrails, the AI might skip the project link or categorize tasks differently each run.

If you want something to behave the same way every single time, make it an MCP server.

Building block 3 - Hooks

Hooks are triggers that fire at specific moments in your conversation with Claude. The most powerful one is the session start hook. Every time Dave opens a new Claude Code chat, the hook automatically injects his weekly priorities, quarterly goals, working preferences, and past mistakes into the context. The AI never starts from scratch.

Hooks are only available in Claude Code terminal and Claude Code desktop. Not in Cursor. This is the single biggest reason to graduate from Cursor to terminal.

Skills are what you do. MCP is how you connect. Hooks are how you compound.

4. The Compounding File System

Here is the fundamental difference between this approach and just chatting with ChatGPT.

In a normal AI conversation, everything lives in that one chat window. Maybe the AI encodes something into its memory. But there is no structured, persistent knowledge base that grows over time.

In a Claude Code operating system, everything is a markdown file. And those files are alive.

How it compounds

When a new Granola meeting transcript comes in with actions from it, or there is a new angle from a stakeholder on a call, that information gets appended to the stakeholder’s person page, to the project’s page, and to the company page if there is a company connected to it. Every time the AI later pulls on that entity, it has all that fresh context. You can ask, “What do you know of this particular project?” The AI knows where to look because the file already holds months of accumulated context.

The mistakes file and working preferences

Dave has a mistakes file that the AI writes to whenever it makes an error. That file gets injected into every new session through the session start hook, so the same mistake never happens twice. He also has a working preferences file that captures how he likes to work. Every time he says, “Hey Claude, why did you do that?” the AI picks it up and logs it.

The Claude MD file

Your Claude MD file acts as a map. (Grab my Claude.MD file here.) Keep it short. Anthropic calls the technique “progressive disclosure” - the Claude MD springboards the AI into other files where deeper context lives, rather than stuffing everything into one giant file.

One trick worth stealing from Dave’s setup: he has a section called “harsh truths for Dave” that the AI wrote after he asked it to audit his system for bloat. That section gets injected into every session.

The more you dance with the AI, the more the files get smarter. That is the whole game.

5. From backlog to PRD to Kanban board

Most PMs stop at one part of the cycle. Dave has the full loop running inside his system.

Step 1 - Collect and rank ideas

Maintain a backlog inside your system. Some ideas come from you. Some come from the AI scanning GitHub repos, Hacker News, and Reddit communities for relevant projects.

Step 2 - Generate a PRD from any idea

Pick an idea from the backlog. Tell Claude to write the PRD. The AI reads the full system context, checks for overlap with existing capabilities, recognizes dependencies, and produces a thorough first draft. Dave’s honest assessment as a CPO is that these are strong first drafts. He would want commercial context and tighter metrics before shipping at work. But for personal and open source projects, he has largely stopped editing them. He calls it “vibe CPOing.”

Step 3 - Manage everything on a Kanban board

When PRDs started piling up, Dave described the pain point to Claude. “I have too many PRDs in flight, I need a way to see what is shipped and what is next.” Claude built the entire web UI in three hours. Each card contains a PRD. Each card has a play button. The AI ranks the cards and tells him what to work on next.

That is malleable software. You have a pain point. You describe it. The AI builds the tool. If it works, keep iterating. If it does not, kill it.

6. Career planning inside your OS

Most PMs manage their product backlog with obsessive rigor and manage their own career with almost none.

Dave built a career MCP server that changes that. It does three things automatically.

Function 1 - Scan for evidence

As Dave works with the AI every week, it listens for evidence of skills demonstrated, feedback received, and outcomes delivered. Granola transcripts, conversations, project outcomes. Everything gets collected automatically.

Function 2 - Run skills gap analysis

Based on his career goals, the system identifies where he is strong and where the gaps are. It maps those gaps against his quarterly goals and weekly priorities. When he runs his weekly plan, it says, “Your thought leadership is strong, but you have not invested in expanding strategic influence across the wider business this quarter. Here are the gaps. Here is what your weekly plan should include.”

Function 3 - Calculate promotion readiness

The system calculates a score based on accumulated evidence. When review time comes, the evidence is already assembled. Not scrambled together the night before.

The career system uses the same compounding principle as everything else. Evidence accumulates over time. Gaps get narrower. And your career roadmap stays connected to your daily work instead of living in a document you never open.

7. How to get started today

Dave’s system is open source. It is called DEX, and it is on GitHub. Or you can use mine (free for founding members).

Step 1 - Clone the repo and run setup

You clone the repo, follow the step-by-step guide, and run /setup. It asks for your name, your role, your company size, and your goals. Then it scaffolds the entire system around your situation. Within five minutes, you are up and running.

Step 2 - Start in Cursor, graduate to terminal

Start in Cursor if the terminal feels intimidating. Do not worry about the messy file tree on the left. Trust the AI to organize things. Then once you are comfortable, move to Claude Code in the terminal. That is where hooks live. That is where compounding lives. On Mac, Ghostty is a cleaner terminal experience than the default.

Step 3 - Use your voice

Do not type. Use Wispr Flow or Speechify. Talking to Claude instead of typing completely changes the dynamic. And be very clear about your goal. Do not tell the AI how to get somewhere. Tell it where you want to end up. Give it a precise, sharp goal, and let it figure out the most elegant path on its own.

There has never been a better time to be a product geek. The PMs who build their own operating systems will compound their advantage every single day. The ones still switching between 15 tabs every morning will keep losing 30 minutes before they even start.

Get Transcript

[Bonus] Takeaway Visual

Where to find Dave Killeen

• LinkedIn

• Pendo

• Dex

Related Content

Newsletters:

1. Claude Cowork and Code setup guide

2. How to use Claude Code like a pro

3. My Claude Code PM OS

Podcasts:

1. Carl Vellotti on Claude Code

2. Rachel Wolan on Claude Code for CPOs

3. Frank Lee on Claude Code and MCP workflows

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify and YouTube.

Brought to you by - Reforge:

Get 1 month free of Reforge Build (the AI prototyping tool built for PMs) with code BUILD

Today’s Episode

Most PMs are using AI the same way they used Google in 2005.

Type something in. Get something out. Move on.

That is not how the best PMs are using it. The best PMs have stopped treating AI as a search engine and started treating it as a team member. One that already knows their product, their writing style, their strategy. One that does not need to be briefed from scratch every single time.

That shift is what today’s episode is about.

I sat down with Lisa Huang, SVP of Product at Xero, the $18 billion finance platform. She built the AI assistant for the first generation Meta RayBan smart glasses. She created Gemini Gems at Google. She has been an AI PM at Apple, Meta, and Google - three of the most demanding AI product environments in the world.

And she gave us a masterclass on three key topics:

1. How to build Gemini Gems and AI projects that actually work

2. What she learned building AI into a wearable device

3. What the future of the AI PM career looks like

Listen or watch now:

Apple Podcast

Spotify

If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

Newsletter Deep Dive

As a thank you for having me in your inbox, here is the complete guide to Gemini Gems, building AI features, and navigating your AI PM career.

1. Gemini Gems - the feature most PMs are ignoring

2. The three must-have Gems for every PM

3. How to build one that actually works

4. What building AI into hardware teaches you

5. Key AI PM Topics in 2026:

   • How to build good AI agents

   • How to measure AI agents

   • The future of the PM role

   • How to break into AI PM

Subscribe now

1. Gemini Gems - the feature most PMs are ignoring

Here is the core problem with how most people use LLMs.

Every time you open a new chat, you start from scratch. Your role. Your company strategy. Your writing style. Your product history. All gone. You are essentially hiring a brilliant contractor and handing them zero blueprints every single time.

Gemini Gems close that gap.

Standard Gemini is your general contractor. Capable. Powerful. But needs a full briefing every single time. A Gem is your master craftsman. It already knows your house. It knows your preferences. You ask once and it delivers exactly what you need.

The same principle applies to Claude Projects and ChatGPT custom GPTs. The name does not matter. The principle does. Stop having generic conversations. Start building personalized AI that holds your context permanently.

2. The three must-have Gems for every PM

Gem 1 - The writing clone

PMs communicate all day. To engineers. To stakeholders. To executives. To customers.

Upload your PRDs, past emails, and team Slack messages. Build a version of your LLM that sounds like you. Use it to draft the first version of everything. You will not lose your voice. You will just get it back faster.

Gem 2 - The product strategy advisor

Feed it your company strategy docs, market positioning, and competitor analysis. Use it as a thought partner when you are working through hard decisions.

It will not replace your judgment. But it will surface angles you had not thought of. That is the product strategy edge most PMs are leaving on the table.

Gem 3 - The user research synthesizer

You cannot be in every interview. You cannot read every support ticket.

Upload the raw transcripts, survey data, and customer support tickets. Ask for synthesis. Ask for key themes. This is the kind of AI-powered customer intelligence work that separates good PMs from great ones.

Your email may get cut off around this point. Continue reading online or in the app:

Continue Reading Online

3. How to Build a Gem That Actually Works

Go to Gemini. Click Gems. Create a new one.

Three things matter.

Step 1 - Write detailed instructions

Not “help me write better.” That gets you nothing.

Write a full page of context. Your role. Your audience. Your format preferences. The more specific you are, the more personalized the output. Think of it the same way you would brief a new hire. Give them everything they need to do the job without asking you twice. (Use AI to help.)

Step 2 - Upload your key context documents

What makes a Gem personalized is not the instructions alone. It is the knowledge. Upload your PRDs, past emails, competitor teardowns, roadmaps. The Gem reads all of it before responding.

One distinction worth knowing. Unlike Claude Projects where you can train the project over time through conversation, Gemini Gems work strictly off what is in the instructions and knowledge files.

So as your context changes, update those files. You can even ask the Gem how it would update its own system prompt based on your conversation. Then paste that update in.

Step 3 - Iterate

Your first version will not be perfect. That is fine. Treat it like a mini AI product you are shipping for yourself. Iterate on the instructions. Iterate on the knowledge. The Gem gets better the more you invest in it.

The biggest mistake is vague instructions. Fix that first and everything else improves.

4. What building AI into hardware teaches you

The first generation AI assistant for Meta RayBan smart glasses launched years before the product hit 4 million sales. That zero-to-one process surfaced constraints most PMs never face.

When you put AI into a wearable, the rules change completely.

Weight. Battery life. Privacy. Bystander concerns. The fact that a fashion company like Luxottica does not move like a Silicon Valley engineering team. All of these compress your design space in ways that a purely software product never would.

And then there is the processing question - cloud or on-device?

Cloud is the default today. But on-device is the future. Once you are wearing a device on your face all day, people are going to want their data staying local. As models get smaller and more efficient, the technical barriers to on-device are dropping fast. Privacy wins over performance every time when the device is that personal.

The lesson for any PM building AI features in any context: do not fall in love with the technology. Understand it deeply. The best AI products live at the exact intersection of what the user genuinely needs and what the technology can reliably do today.

Build fast. See what users do. Update your assumptions. Repeat.

5. Key AI PM Topics in 2026

Since Lisa is such a tenured AI PM leader, I had to ask her about all the hot AI PM topics right now.

5a. How to build an accurate AI agent

Here is what makes building AI agents in any high-stakes domain genuinely hard.

Accuracy is not a nice-to-have. It is the product. In finance, the decimal matters. In legal, the clause matters. In healthcare, the dosage matters. And LLMs out of the box are not naturally great at any of these.

Two things close that gap:

Advantage 1 - Domain knowledge

The companies winning at agents are not just wiring up a generic LLM. They understand every workflow. Every stakeholder. Every acceptable accuracy level at each step.

You have to craft the agent experience around your specific constraints. What are the tasks? What are the subtasks? Where does a 90% answer suffice and where does it fail the user entirely? Map that before you build.

Advantage 2 - Proprietary data

Generic agents give generic answers. The agents that win are the ones with data nobody else has.

Transaction-level data. Interaction history. Domain-specific corpora. That data lets you personalize to each user’s actual situation. It also lets you surface benchmarks and insights that a general-purpose model simply cannot.

On architecture: use a hybrid system. LLMs in multi-agent workflows where it makes sense, but programmatic code where you need tighter control over reliability. Not everything should be non-deterministic. Know when each applies.

5b. How to measure an AI agent

This is the question I get most from AI PMs right now. How do you measure something that is non-deterministic and evolving fast?

Three layers. They build on each other. Do not skip ahead.

Layer 1 - Quality

Is the AI doing what it is supposed to do?

This means evals. Human annotators. LLM judges. You need all three because none of them alone scales. Human annotators give you ground truth. LLM judges give you scale. Evals give you a consistent framework across use cases.

Track quality regularly across all use cases. Understand the gaps. Know which investments close which gaps. This is your foundation. You cannot build reliably on top of it until it is solid.

Layer 2 - Product metrics

Once quality is solid, standard product metrics apply. Adoption. Usage. Retention. CSAT. MAU or WAU depending on the use case.

Also track qualitative signals. Social media. Customer conversations. Support tickets. Users will tell you things in those channels that no dashboard will surface.

Layer 3 - Business impact

Revenue attribution. Retention influence. ARR contribution.

Every company measures this differently. What matters is that you have a system for it and that you track it consistently. AI investment needs to show up somewhere on the business scorecard.

The order matters. Quality first. Product metrics second. Business impact third. Skip to layer three without the foundation and you are measuring on sand.

5c. The future of the PM role

A lot of PMs I talk to are anxious about this. I hear it constantly.

The layoffs feel disproportionate. Junior PM roles are harder to get. The director and group PM layers are compressing. The question I keep getting is: is this career still worth pursuing?

Here’s what I’d say:

AI will not replace PMs. What it will replace is the execution work that fills most of a PM’s day today. Writing PRDs. Creating mocks. Managing roadmaps. Pulling data. All of it will be accelerated or automated.

But product judgment? The ability to look at ambiguous signals and say this is the right bet and here is why? That is not going anywhere. That is what PMs get paid for. Not the deliverables. The taste.

What is changing is the structure. PM-to-engineer ratios will compress. The pure PM role is evolving into a hybrid. The expectation is becoming that PMs also build. Not just spec and hand off, but prototype, design, and code enough to show what they mean. As Lisa said:

Now is a time of transformation and everyone has the availability to go do that.

5d. How to break into AI PM

The excuse I hear most is “I don’t work on AI features at my company.”

It is not an acceptable excuse.

You do not need your company’s permission to build AI products. You do not need a budget. You do not need a team. Claude, OpenAI, Gemini, all of them give you access to the same models that companies are building on. Most companies are not fine-tuning anything. They are using the exact same consumer tools you already have access to.

So build. Build Gems. Build projects. Build small AI products that solve problems you actually have. Use your personal data so there are no company restrictions.

How to stand out in an interview

One candidate who got hired for a senior AI role had zero AI experience going in. Lisa was not sure about them. Then in the first interview they said:

“I saw you are building financial tools for small businesses. I went and watched three hours of TikTok videos from coaches who work with small businesses. Here is what they said about what those businesses actually need financially.”

Nobody else had done that. Not one candidate. You can guess who Lisa hired.

Do the work before you are asked to. That is the whole strategy.

The roadmap to break in

Get direct AI experience in your current role if you can. If not, build on the side. Invest in your network because referrals still matter more than most people admit. Show up well in every job because today’s colleagues become tomorrow’s connections.

When you are ready to interview, treat interview prep as its own skill. Practice out loud. Get mock interviews with people who have been inside those companies. Drill until the structure is second nature.

The FAANG PM interview is a specific game. Product sense. Product execution. Behavioral. Case questions. You can be a great PM and still lose if you cannot perform in that format in that time window. Practice the format, not just the content.

Get Transcript

Where to Find Lisa Huang

• LinkedIn

• Xero

Related Content

Newsletters

1. AI product strategy

2. How to become an AI PM

3. Practical AI agents for PMs

Podcasts

1. Claude Code + Analytics - Vibe PMing with Frank Lee

2. AI evals explained simply with Ankit Shukla

3. How to become an AI PM with Marily Nika

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify and YouTube.

Brought to you by:

1. Pendo: The #1 software experience management platform

2. Testkube: Leading test orchestration platform

3. Gamma: Turn customer feedback into product decisions with AI

4. Product Faculty: Get $550 off the AI PM Certification with code AAKASH550C7

5. Mobbin: Discover real-world design inspiration

Today’s Episode

Every forward-thinking product team is using AI prototyping tools.

But most are using these tools wrong.

They open Bolt or Replit, type a vague prompt, get something that looks decent, show it around, and move on:

• No problem space work

• No divergent solutions

• No user testing

That’s why I’ve created today’s episode. I got Nadav Abrahami (Co-founder of $5.5B giant Wix, CEO of Dazl) to walk you through the complete workflow to use these tools to their maximum capability.

I’ve done three episodes on AI prototyping. This one builds on that compendium of knowledge to cover: when to prototype, how to prompt, when to go high fidelity, and how to hand off to engineers with zero open questions.

Listen or watch now:

Apple Podcast

Spotify

If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

Newsletter Deep Dive

As a thank you for having me in your inbox, here’s a complete guide to AI prototyping for PMs:

1. When to use AI prototyping

2. The ideal workflow: step-by-step

3. How to prompt without breaking things

4. When to go high fidelity vs low fidelity

5. PRD + prototype - the new standard

6. How to hand off to engineers

Subscribe now

1. When to use AI prototyping (and when not to)

Let me start with what I see happen constantly.

A PM finally gets access to an AI prototyping tool. They start using it for everything. They produce a lot of prototypes. Zero of them go anywhere.

The tool isn’t the problem. The sequencing is.

AI prototyping tools don’t replace problem space work. They accelerate solution space work. Those are two completely different things.

Before you open any prototyping tool, you need three things locked down:

1. The problem you’re solving

2. The user story you’re designing for

3. The rough shape of the solution

If you can’t write all three in a single paragraph, you’re not ready to prototype. Go back to discovery.

Once you have those three, there are two moments in the development lifecycle where AI prototyping really shines.

Moment 1 - Early ideation

This is where most teams barely prototype at all. A rare ~5% of exceptional PMs do it at the idea stage.

That’s a huge missed opportunity. Running 3-4 variations at ideation takes the same time it used to take to run one. You’ll learn more from 30 minutes of functional prototyping than from two hours of writing user stories.

Moment 2 - Post-Figma validation

After the design team has narrowed on a direction, you want to get as close to the real experience as possible before handing off to engineering.

Static Figma screens can’t tell you whether the interaction actually works. A functional prototype can. It’s also ~75% of forward-thinking teams’ most common use case for these tools right now.

Do the research first. Then prototype.