Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by

Mobbin - The internet’s best library of design patterns

Check Out Mobbin

Get a full year of Mobbin Pro, along with 8 other AI tools, at bundle.aakashg.com.

Today’s Episode

Billions of people have played Mark Pincus's games. He founded Zynga, and for years I walked past the Zynga building in San Francisco every day.

Mark just published Life at the Speed of Play. The book's subtext is how to launch products people love.

So I asked him to go to the whiteboard and teach his Proven Better New framework from scratch.

I use this framework myself when I think about building products. But I'd guess only 10% of you know it well.

By the end of this post, you will know:

• Why every great product starts as a copy

• Where the line is between stealing well and getting it wrong

• How to sort your roadmap into proven, better and new

🎁 Win a free copy of Mark’s book

I’m giving away 10 copies of Life at the Speed of Play! All you gotta do is watch the episode on YouTube and comment I watched till the end. I’ll reach out to 10 of you.

Go Comment!

Want to see an AI agent get built from scratch?

This Wednesday, Oct 28, I’m building one live in Claude Code and explaining MCP in simple language.

Register Now!

Newsletter Deep Dive

1. How to use Proven/Better/New to Build Better Products

2. How to tell better from new

3. When to ship less new

4. How to test 100 ideas before you build one

Subscribe now

1. How to use Proven/Better/New to Build Better Products

The big online poker sites used to make you download before you play a single hand.

Mark noticed that the moment you ask for a download, you lose at least 80% of your audience. That's his number, not a benchmark.

So he copied the leading poker tables down to the chip colors and removed the download.

Then he added the one new thing: your Facebook and MySpace friends could sit at your table.

Mark's most controversial idea is that copying is okay.

All music, movies, stories and games are derivative. Every artist builds on the artists before them.

But there is a line between taking inspiration and copying blatantly. That line comes down to taste.

Steve Jobs was a master of copying and appreciated it deeply. Bill Gates copied from the same place, Xerox PARC.

Then the two copied each other. Gates was worse at the trick, so it looked like he was just copying Apple.

“can you learn from other people's proven success and emulate it perfectly, but have it feel new to people?”

Proven Better New lets you act on both at once.

Proven is any part of the experience that already works for this audience on this platform, so you copy it click by click and pixel by pixel.

New is your novel idea and your innovation zone. Your job is to isolate it so your launch measures that idea and nothing else.

If you want the first-session side of copying what works, my Ultimate Guide: Activation (paid) and The Ultimate Guide to Onboarding (paid) go step by step.

2. How to tell better from new

Would 10 out of 10 people who use the proven product today choose it? If they all agree, call it better.

If only you think so, call it new. This test isn't always easy, and most mistakes go in one of two directions.

The first is calling your own idea better. Take Mark's family assistant built to compete with Instinct, the AI assistant that works over text. I tested Instinct for AI by Aakash.

Putting the whole family in one thread feels like an obvious improvement to the person building it. Under the test it is new.

Mark left the Better column empty:

“I have no ideas today that I could say 10 out of 10 Instinct users would call better than Instinct.”

The second is calling a removal better. Rejecting something that looks like an issue feels like progress.

But if it already works, removing something proven is a bet like any other. Zynga copied the neighbor bar from FarmTown, where the other player has to accept your request.

It looks like friction. Mark says calling its removal better “would have made our product less viral.”

The loop behind that is in my Ultimate Guide: Growth Loops (paid).

People at Zynga still repeat his mantra that all new fails.

If you expect that from the start, you won't be disappointed or let a failing idea linger too long.

But when new does work, we need to have a reverence for it.

So discount the hope you have for your own new idea. And pay close attention to new that's working out in the market, because that new just became proven.

Consumer AI didn't feel investable to Mark back in June. With Instinct at a $10B valuation, it feels a lot more investable today.

3. When to ship less new

Mark's proof is Hay Day. Supercell's game looked like Zynga's FarmVille 2.

In his words, “it was 0% new.” What changed was the better: it loaded faster, it had shorter crop cycle times and it had touch.

Late in a cycle it flips. Nobody wants a new farm game now, and you'd need “a lot of new.”

AI is early. So a lot of low-hanging fruit is always going to be there.

It is best to be disciplined about it and look for where you can be better because of AI.

Write down which part of your AI product is actually new. If it's more than one thing, you're running two experiments in one launch.

My guide to AI strategy is How to Build Product Strategy in the Age of AI (paid).

4. How to test 100 ideas before you build one

AI lets you make an MVP in months instead of a year and a working version in minutes.

Mark thinks that speed is a trap, because it tempts you to skip the top of the funnel and go straight to the whole product.

So you end up failing too slowly.

He had just argued about it with Eric Ries, whose Lean Startup gave us the MVP: “the problem with MVP is the word viable.”

Once you call something viable you start thinking it could be your launch product. I made a similar point in 2022 in “MVP”: the most misused term in tech.

The MVP is the version that tests your most critical assumption. Mark's funnel test goes one step earlier and checks demand before there is a product at all.

I go deeper on scoping in Should your MVP be Embarrassing? (paid).

His proof was a post by Rohan Nayak, CEO of Pocket Entertainment, the company behind Pocket FM.

Pocket FM runs a 90-second video trailer of an audio drama as its ad.

Mark's reading is that once they hit 2.25% in any country, they could grow to $10 million in ARR in three months.

Rohan's own numbers are narrower: France to $10M in 3 months, and 8 months to figure the engine out.

Mark says he could “vibe code something at least good enough to be the website behind an ad” and send an agent into France, Italy and Saudi to see if it gets a 2.25% click-through rate.

Here is how to run it:

1. Write the teardown. Fill in the proven slots for the closest competitor at click-level detail.

2. Generate concepts. A batch of names and one-line value props for each bet. Ask the agent to flag where it needs you to brainstorm.

3. Vibe-code one page that's just good enough to catch an ad click.

4. Run web ads. Mobile ad platforms may not let you advertise without an App Store listing.

Pocket FM's own benchmarks from Rohan's post are 2.5% CTR and 55% 3-second through-play on a 90-second trailer. Pick yours from your own funnel.

Get Transcript

Try this first

Open your roadmap and write proven, better or new next to every item. Anything you can't point to a competitor doing is new.

If one launch carries more than one new, you've found your first cut.

Where to Find Mark Online

• X: @markpinc

• Life at the Speed of Play: Launch Products People Love!

• Read the full transcript

Related Content

1. The Ultimate Guide to Onboarding (paid): the first clicks of a new user

2. Ultimate Guide: Activation (paid): getting users to the good stuff

3. Ultimate Guide: Growth Loops (paid): the loop behind the neighbor bar

4. How to Build Product Strategy in the Age of AI (paid): what to build as the platform shifts

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by

• Fingerprint - Identify good and bad visitors with industry-leading accuracy

• Ariso - Ship AI agents and features faster, with fewer regressions

• Customer.io - Send smarter messages using your product data

• Amplitude - The market leader in product analytics

• Product Faculty - Get $150 off their #1 AI PM Certification via code AAKASH150

Today’s Episode

Claude Code can be the most powerful system in the world.

I’ve now built you several Operating Systems you can run out of it: for content, for PM, your team, and for your company.

The missing piece was a Growth OS. So today, I present you one. And I haven’t just ported over what works for me.

I’ve brought on James McAulay who worked on growth at ElevenLabs and grew his latest business, Agent Accelerator, to £100K a month in 95 days, without hiring a single FTE.

He’s open-sourced everything. And it works.

Halfway through our recording, James ran a Claude Code skill on my site. A few minutes later it returned posts with no SEO description, long titles, crawlability issues, duplicate content…

(Which I’ve now fixed, with this growth OS).

Free Webinar

The technologies that make a growth OS possible are AI agents and MCP. I’m giving a full masterclass on them for free next Wednesday:

Join the Free Webinar

Newsletter Deep Dive

I appreciate all the support you give my work. So here’s a written deep dive if you prefer it:

1. How the Growth OS Works

2. How to Set Up Your Growth OS

3. How to Run Growth Experiments with No Engineer

1. How the Growth OS Works

You open Claude Code inside it, and it has three layers: tools with APIs, everyday skills, and routines that improve the setup.

Tools with APIs

James only adopts a tool if it has a good API or CLI, like Stripe, PostHog, Attio, Firecrawl, DataForSEO, vidIQ, Kit, Apollo

Everyday skills

Morning standup, weekly and monthly retros, PRD skill, SEO audit, tone-of-voice, YouTube packaging, competitor scan

Routines that improve the setup

The system improves itself from your usage, as weekly scheduled jobs look at how you work. It even suggests new skills and other connections you might want to add.

The repo is growth-agents on his GitHub. It installs with a couple of commands and runs in Claude Code, Codex, Hermes and OpenClaw.

It’s built for a full-stack growth person or a company’s first growth hire. If you’re a specialist (a product marketer or SEO lead), the onboarding asks about your goals and adjusts the system to your role.

2. How to set up your Growth OS

So you don’t actually need a new folder for this. If you’ve got a second brain already, the folder where all your notes live, just put the OS in there.

Clone the repo and run the install commands from the README. Then open Claude Code. Run the onboarding and Claude goes through everything you’ve already got in there before it asks you a single question.

It’ll ask if you want rapid or full setup. James makes everyone he teaches do the full two hour setup. We did rapid on camera because we were short on time. Try the full one on yours if you can.

What to wire into Claude Code

Connect everything as fast as you can.

The two most important are analytics and Git:

1. We did Posthog via MPC.

2. Read-only access to Git is key. Your team might not let you ship code, but it helps you get the right PRs ready.

The OS will help you so never forget PostHog events again. Every event says when it fires, where it fires, and what to measure with it:

Pre-AI, you’d only see data that clean under a god-tier PM. Now, it comes with the OS.

The core loop: actually learning from experiments

Running a test is easy but concluding it with definitive learnings is a feat unknown to many. I know it may sound harsh but shiny-object syndrome pulls us onto the next big idea, and at year’s end we are left with a pile of PostHog experiments with no recorded learnings.

The OS solves for the exact problem:

• Every experiment gets a file with these fields before anything ships.

• A playbook for A/B tests is set up, and it stops you from reading a result early. The weekly retro and the daily standup both read the list of running experiments and tell you when one is ready to review. They also catch stale tests.

• Experiments only build on each other when a template forces the baseline and confounders up front and a weekly loop forces the learning at the end.

3. How to run growth experiments with no engineer

The way growth was done before, you’d wait on an engineer before any test could happen. Now growth people can run experiments themselves. That’s how James thinks about it.

We no longer need to wait for engineers to build growth experiments.

Gate 1 - PRD

Say you want a new page on the marketing site. What the system does is help you have the discipling to write a PRD where you answer key questions like: What metric will it move? So use the PRD skill to write a draft.

Gate 2 - Multi-agent refine

Then refine it with the multi-agent refine skill. This is cool. It launches subagents to critique your PRD, each using a different angle. Then a lead agent creates a master critique and priority list of action items to do.

Then - Ship

THEN ask Claude Code to build your feature.

Example: SEO

Here’s the whole loop in action with SEO.

Start with the SEO audit skill:

It’s a remix from Corey Haines’s Marketing Skills collection. But James added a few tools that really upgrades them:

1. Firecrawl which turns any website into Markdown, because when you ask Claude to scrape a site, it often hits Cloudflare blocks or misreads the page. Firecrawl fixes that.

2. DataForSEO is one of my favorite finds for this episodes. It’s a super cheap SEO API. Once you give Claude Code the key, it can research any term for you, just for a few cents.

3. The Lighthouse CLI runs Google’s page speed tests from the terminal. Before, you’d open PageSpeed, run the test, export it, save it and hand it to the agent yourself.

What it found on my site was that my search traffic was about 25K visits in May and has dropped to about 10K.

Then, it can help you ship the changes. I had it execute on this whole action plan:

Try it yourself! It’s pretty magical.

Final Words

How autonomous are growth agents, really?

Is the ‘autonomous’ part overhyped? Mostly, yes. A lot of the people on LinkedIn and X claiming autonomous agents are either burning tokens on low-value work, or running an agent that does most basic stuff.

But, on the other hand, people also don’t realize an agent can improve itself. That part is autonomous, and really cool.

So go use the Growth OS, push on its autonomy, but don’t expect it to work without you.

Get Transcript

Where to Find James Online

Please, let’s follow him everywhere and hit his businesses with a flood of Aakash’s podcast listeners!

• LinkedIn

• X

• YouTube

• Website

• Agent Accelerator

• GitHub

That’s how we’ll keep getting bigger and better guests. Talk soon.

P.S. An annual membership gets you a year of paid plans for my favorite AI tools at bundle.aakashg.com.

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by

• Fingerprint - Identify good and bad visitors with industry-leading accuracy

• Ariso - Ship AI agents and features faster, with fewer regressions

• Customer.io - Send smarter messages using your product data

• Amplitude - The market leader in product analytics

• Product Faculty - Get $150 off their #1 AI PM Certification via code AAKASH150

Today’s Episode

On February 28, Pieter Levels posted that for the first time in his life, he thought he’d outrun his to-do list. With Claude Code doing the building, the bug board his users post to was empty.

I’d say this is about 10x my normal output, he wrote about the week’s shipped list. Over 1 million people viewed it. His last line:

The real bottleneck is becoming myself and my creativity, not how fast I can ship.

John Cutler ran into the other side of it. He wrote that CEOs now claim their teams are 100x more effective. He asked one, who runs a collaboration tool, whether there was 100x more demand for their product. The answer: “Crickets.”

That’s where most product teams are right now.

Output is up, and the open question is whether any of it made a difference. Or as today’s guest put it on the show: When features get cheap, not having chosen the right feature, funnily enough, gets more expensive.

So today I brought in the strategy guy, Tim Herbig (10+ years as a PM and Head of Product, now coach to product teams), to spill

• How do you tell a good strategy from a list of priorities

• What role should AI play in your product strategy

• And which metrics prove it’s working

He walks through all of it on a live board, with one made-up company that stands in for many of his clients.

Two FREE Resources!

The podcast is, of course, totally free (as is the newsletter deep dive below). I have two more resources for you, as well:

1. A Free Webinar on the Job Search

Want more interview callbacks without sending more job applications? Join me for a free live class this Friday, October 16, at 9:30 AM PT / 12:30 PM ET:

Join Webinar

2. A Free Conference to Up-skill in AI

I’m giving a talk on the “Future of the Full-Stack Builder” and moderating a panel on the future of product in two weeks, Thursday, October 29, at 9AM PT / 12PM ET:

Attend the Conference

Everything I do is for you. I hope these free resources help you in your career.

🎁 And Contest to Win a Free Book!

I’m also giving away 5 copies of Tim’s book, Real Progress. It’s just 3 steps:

1. Subscribe on YouTube and comment on the episode

2. Follow the show on Apple or Spotify

3. DM me a screenshot of both on LinkedIn

The first 5 people get a FREE copy!

Newsletter Deep Dive

Now… to our main event. As always, I’ve written up the podcast as well. That’s how much I value being in your inbox. Here’s what we’ll cover:

1. What product strategy is actually for, and where AI fits

2. How to test whether you made the right choice

3. How to choose differentiators that survive AI

4. How to know if your strategy worked

Subscribe now

1. What product strategy is actually for, and where AI fits

A few years ago, the limiting factor for most PMs was time. Building took a long time, so you had to prioritize ruthlessly. Today, even established companies get stuff out the door much faster.

But you still have to get people to use what you ship. Tim says many of his clients told him a year ago that feature adoption was already a problem.

So sooner or later, every team amping up its output has to ask: did any of it make a difference?

Our mutual friend Ravi Mehta (former CPO at Tinder) says difficulty prioritizing is often a strategy issue, not an execution issue.

So you need to start there.

Who is it for, and how would you know it worked? It’s for your teams and stakeholders. And it works if it helps them intentionally say yes or no to opportunities over a set horizon, say 6 to 18 months. You know it’s working if most of your team’s energy goes to work connected to the strategy.

As a product leader, I was once handed a strategy with 14 priorities. So what makes a strategy able to say no?

• Strategic narrative

  Why should we act, and why now?

• Playing field

  Who are we solving problems for, which problems, and what do they use today instead of us?

• Winning moves

  What value do we provide, and how will we stand apart?

Your answers have to be opinionated enough to enforce a no.

Tim’s view of canvases and templates: they’re windows into the actual work of strategy. The fundamental questions don’t change. Answer them first, then assemble whatever shape your boss wants.

Where AI fits (and where it doesn’t)

It would be stupid if we didn’t use AI as a tool for making strategy. Researching markets and competitors; synthesizing customer evidence; generating and comparing scenarios; pressure-testing your bets. But you’re still accountable for the consequences.

Treating AI as the value proposition is wrong. Almost no technology in history has been one on its own. AI is a resource you use to amplify the core value customers already come to you for.

Everyone has access to the same frontier models, but you can’t compete on that alone. I pushed back with 2 counterexamples. Pieter Levels’ Photo AI, which does $83K a month in revenue as a one-person company. And ChatPRD, which Claire Vo built while she was a CPO. Isn’t AI the whole product there?

Look at what’s layered on top. People root for Levels because of his personal brand, which keeps him out of a head-to-head with the big companies. Tim’s bet is that people pick ChatPRD for Claire’s product judgment built into it. That matches what I wrote in my AI product strategy guide in March.

2. How to test whether you made the right choice

A favorite pressure test comes from Roger Martin, who I had on the show for a strategy masterclass. Once you see it, you can’t look at strategy the same way.

If the opposite is stupid on its face, you haven’t made a choice.

Tim’s example: “make salespeople more efficient.” The opposite is slower salespeople, which is stupid, so that isn’t a choice. “Make salespeople more efficient by being in person with clients” passes, because virtual contact is a real alternative. One of Tim’s clients calls this the stupidity flip.

For example, sellers and dealers want qualified buyers who buy fast. Buyers arrive from Google and AI search. Many are first-timers who don’t trust a car they haven’t seen. Fair Mile (a hypothetical used-car marketplace) steps in to handle inspections and build trust.

Its default strategy is to paste AI everywhere. That’s an all-yes strategy that says no to capital, margin, and quality.

Of the seven things Tim flips, only 2 are real choices. And they’re the operational ones, where the company picks a mechanism (human call-center verification vs. automated, a static start page vs. an AI-personalized one). The headline phrases fail because no company would argue the opposite.

When I was at Apollo, the strategy from our CEO was to only focus on paid users. But as a product leader, I couldn’t just ignore free users. So how myopic should a strategy be?

First, a strategy doesn’t have to cover everything you do. Focusing on paid users doesn’t mean turning the product off for free ones.

Second, the choice has to fit the narrative. Why paid users? If the narrative is to grow revenue by improving retention for paid users, then a push to acquire more free users is a contradiction.

That was exactly our reasoning at Apollo. Retention was the biggest lever.

Strategy isn’t a top-down thing. You can start anywhere. Someone hands you a segment, and you ask why. Someone hands you a differentiator, and you ask who it’s for. The answers don’t have to be right, because no one knows what’s right anyway. They have to reinforce each other.

Once the audience is set, everything downstream narrows. Tim borrows the audience tree from Andy Johns, a former growth leader at Facebook, who said your target audience should be comically narrow. Tim’s guardrail: land somewhere between very broad and comically narrow.

The audience tree works like this. Start with the broadest audience and a comically narrow one, then list the ways you could slice it: region, income, car value, first-time or repeat buyer. For Fair Mile, Tim lands on repeat buyers of cars worth more than 100K in Germany. Germans love cars and often buy new ones. Repeat buyers might be easier to convince to buy online. And above 100K, every transaction carries a big margin.

This is where AI helps. Ask it how many car owners there are in Germany, how often they buy, what they pay, and how many cars competitors list above and below 100K. Tim’s illustrative answer was about 50,000 people you could reach every year. If the number is too small to fund the company, widen it. Germany becomes Germany, Austria and Switzerland.

3. How to choose differentiators that survive AI

Tim calls differentiation the trickiest component, and the one AI changed the most. He doesn’t invent a new framework for it. He goes back to Hamilton Helmer’s 7 Powers, the classic list of sources of lasting advantage.

For each power, he asks

• What does it do?

• Can we actually execute it?

• Does it help us differentiate from our competition?

Notice the pattern in what survives. The durable powers are the ones that take the longest to build. You can’t speed up a brand or a reputation. You can destroy one incredibly quickly.

Counter-positioning is worth a second look, because the classic example is so clean. In 2000, Blockbuster collected nearly $800 million in late fees, 16% of its revenue. Netflix offered subscriptions with no late fees. Blockbuster couldn’t follow without breaking its own business model.

For Fair Mile, Tim picks brand and network effects, with dealership scale as a maybe. Switching costs don’t fit a car marketplace. And he would skip a fancy AI search: “No one will buy a car with you because you have an AI search on the marketplace.”

So if there’s an order: audience first, then powers?

The truth is, you can start anywhere. Joe from marketing walks in and says, We now differentiate on price. Fine, follow the link. Is our audience price sensitive? Why do customers pick competitors today?

And if the answer is price, get out of that market. You can’t win on price. Tim’s only exceptions are companies like the big AI labs, which can subsidize price for a while. You don’t have that luxury.

4. How to know if your strategy worked

When teams hear metrics, they jump straight to the framework. Instead, answer 2 questions in plain English first:

1. What will you do differently?

2. How will you know it worked?

This applies to B2C, B2B, and internal teams alike. AI teams usually reach for metrics like tokens used or output. Tim’s line: tokens used is not a value exchange for your customers. It’s a value exchange for Anthropic.

So ask whose behavior should change, and by how much. If the goal is 20x more AI feature output next quarter, ask what changes for whom as a result. Do you want your team more AI-fluent? Or customers solving their problems faster? Make that the goal.

The same goes for the North Star metric. Teams think it is EBITDA, but it’s not. Tim credits Sean Ellis with the concept. Ellis defined it as the metric that represents the value customers experience from your product. When it rises, business metrics should follow. The business metric itself is something else.

If you’re burning tokens to ship faster but still need months to see customer behavior change, what exactly is the point of that?

You can also use Ravi’s product strategy stack. Think of each level as a funnel. For every choice from the level above, ask whether it already lets me say no.

Before setting your strategy, ask what the company is already incredibly good at, and how your area can build on it. Personio calls these evergreen benefits.

If the company has mastered a distribution channel or has integrations in place, don’t reinvent them at the feature level.

Only then connect the roadmap. Every roadmap item should map to one of your strategy’s metrics.

Get Transcript

Where to find Tim Herbig

• LinkedIn

• Website

• Product Practice Newsletter

Share

And that’s all for today. Talk to you soon :)

P.S. You can get a year of the paid plan’s of my favorite AI tools with an annual membership: bundle.aakashg.com.

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by

Mobbin - The internet’s best library of real design patterns

Check Out Mobbin

Get a full year of Mobbin Pro, along with 8 other AI tools, at bundle.aakashg.com.

Today’s Episode

Srini Raghavan, the CPO of Freshworks, predicted that the title “Product Builder” will replace engineer, designer, and product manager in 5 years.

If that’s true, we all have a lot of catching up to do.

So today I break it all down with one of my most popular guests, Ankit Shukla (founder, HelloPM):

• Is the Product Builder role legit…

• How much are Product Builders paid and where do they work…

• And what you need to know to become a product builder

Want a free live class covering the AI you need to know to become a product builder?

Join me tomorrow morning at 9AM:

Join the Webinar

Newsletter Deep Dive

I am so glad you let me in your inbox :)

As a thank-you, I wrote this complete guide to becoming a product builder. Enjoy!

1. What even is a Product Builder, who’s hiring, and does it pay?

2. How to find AI problems that are actually worth solving

3. How to ship a production app with Claude Code

4. How to land a product builder role

Subscribe now

1. What even is a Product Builder, who’s hiring, and does it pay?

What Even is a Product Builder

The big question on the role “Product Builder” right now is:

Ankit’s take? It’s real.

In the episode, Ankit pulled 12,500 PM job descriptions from top companies and had Claude and GPT read them. More than 30% of PM roles now ask for AI skills.

The number one skill those postings asked for was judgment: knowing which problems deserve AI.

So that judgment is what people are paying for.

- Ankit

Indeed, LinkedIn shut down its Associate Product Manager program and replaced it with an Associate Product Builder program.

So how does this differ from other jobs out there?

The two commonly confused ones are forward deployed engineer and product builder. Here’s the difference between the three:

Put simply:

• AI PMs are the PMs building AI models or features

• Forward Deployed Engineers are the on-the-ground consultants helping build deployments of an AI product for specific customers

• Product Builders are, today, any PM building with AI. Some believe it becomes the one title for design, engineering, and product management.

How Much Does it Pay

Right now, product builders really hold the title AI PM.

Looking at those roles, AI-skilled PM roles pay 15 to 20% more than traditional ones in the US and Europe, and 30 to 50% more in India.

How this breaks down in the US and India is:

In the US, Ankit found:

• $195K median

• Around $120K base if you’re just starting out

• $340K to $350K in senior roles

• Up to $2.5M in total comp for senior roles at the top labs

In India:

• ₹12 to ₹16 LPA for a fresher with solid AI projects

• ₹22 LPA at 2 YoE

For context, our survey of 1,019 Indian PMs put the average APM at ₹17 LPA.

Who’s Hiring

These days PMs as product builders is the hottest topic. Here are just three product leaders I recently chatted with who are hiring this archetype:

• Vishnu Gopalan, VP of Product at Series B Opsera

• Jiaona Zhang, CPO at Series C Laurel

• Srini Raghavan, CPO at Freshworks (public)

Many others I talk to have emphasized the same. I’d say this is the biggest trend in PM right now. So when in doubt, assume the product builder skillset is what any AI-forward company you interview with is looking for.

2. How to find AI problems that are actually worth solving

The most important skill every product builder needs to have is great AI product sense. They need to be able to find AI problems that are actually worth solving.

Here’s the framework Ankit drew out for us, POWER:

Let’s go deep on each letter.

P - Possibilities

Start from what AI can do. Ankit’s shorthand is UTG:

• Understand: read a whole folder of meeting transcripts

• Transform: turn those transcripts into one summary

• Generate: write the content or the code

Then go see what other people did with it. Your competitors. The industry next door. And his favorite shortcut, the customer stories pages at Anthropic, OpenAI, DeepSeek and Moonshot. Each one is a list of things a real company already made AI do.

Write all of it down. Ankit calls it a possibilities database, and says every company that’s serious about AI should keep one.

O - Opportunities

In possibilities, you looked at the world. In opportunities, you look at your company.

Two rules here:

1. Never start with one use case. You’ll pick before you’ve seen the whole map, and you’ll usually pick wrong.

2. Pick by frequency. Build for a rare problem and it never pays back the time it took.

This is the judgment Ankit kept coming back to all episode:

“For you as a product manager, one skill that is very important is about judgment of what problem to pick in order to solve with AI.”

W - Workflows

Ankit says this is the step where PMs need to put the most attention.

“Improve discovery with AI” sounds like a plan. It’s too high-level to build anything from.

You have to go down to the actual steps. Take a roadmap decision. The inputs are:

• Support tickets

• Reviews on Google Play or G2

• Sprint reviews

• Leadership meetings about the product

Where does each one go today? Who reads it? What gets dropped? You only find out one way: interview the people who do the work.

Then sort what you find into two buckets:

• Optimization: the three-hour weekly task now takes one

• Innovation: a process that couldn’t exist before, so nobody ran it

Innovation is where it gets fun.

E - Engineering

Notice something? Three letters in, and not one tool named.

That’s the point.

Only now do you pick: n8n, Make, Zapier, a Gem, a custom GPT, Claude.

Start from a real use case and you get ROI. Start from a tool and you spend your time retrofitting a problem to fit it. Ankit says that’s how the AI slop gets everywhere.

And here’s the stat I keep thinking about. Across those 12,500 job descriptions, the number one skill wasn’t RAG. It wasn’t agents or prompt engineering either.

It was identifying the right use case for AI.

R - Reflection

Ankit wrote the R on the whiteboard but didn’t get to it on the show. HelloPM’s own notes on POWER fill it in: R is for Reflection.

“After the AI does its work, you have to look at what came back, find the gaps, and improve your approach for next time.”

That’s what makes AI work compound instead of starting from zero every time. It’s also where evals come in. My guide to your first eval is the place to start.

3. How to ship a production app with Claude Code

Say POWER worked. You found a problem worth solving. Now you’re at E.

And the internet has a dozen opinions. One thread says Claude Code. Another swears by Codex. Your engineering lead wants n8n. A creator you follow built the whole thing with a Gem.

So most PMs either overbuild or underbuild.

Ankit’s answer is a ladder. Six levels, from a plain prompt at L0 to a production app at L5. Each level fixes a pain in the one below it, and each one costs more to build.

Claude Code or Codex?

Everyone asks Ankit this. So his team tested it.

They built the same four sites (an ecommerce store, a social commerce site, a social network and a gaming platform) with Claude Code, Codex, Cursor and a DeepSeek setup.

Every one of them shipped a working site.

“For 90% of the use cases, it does not matter.”

Stop debating the model. Start building.

4. How to land a product builder role

So where do you start? With the same roadmap one of HelloPM’s students just used to land a senior PM role at Sarvam AI (India’s latest unicorn):

Step 0 - Align

You can’t land a job you don’t understand.

So pull 25 to 30 job listings. AI PM roles, or PM roles at AI-native companies. Banks, fintech and health tech count too. They’re all asking for AI now.

Read the qualification lines, not the titles. When I read 113 AI PM listings by hand in August:

• 7 in 10 wanted prior AI experience

• Only 1 in 8 wanted it in production

• Plain “Product Manager” listings asked for AI almost as often as “AI PM” ones

Then build your skills map. By hand. Ankit was firm on this one. Reading job descriptions closely is where humans still have the edge.

Last, name your edge. Ankit says nobody with two years of experience has zero:

• BA or product owner: stakeholders and the SDLC

• Marketing or sales: customers, distribution and usually a domain

• Engineer: solutions and technology

Your background is already a head start. Write it down.

Step 1 - Acquire

Take each skill on your map to ChatGPT or Claude for a brief. Watch the videos. Now you have information.

“Information actually gives you a false sense of confidence that is shattered the moment that you enter the interview.”

You need knowledge. Ankit says it comes from two places.

Build something. Run POWER on a real problem and climb the ladder. A live build you can defend (the model choice, the evals) clears most of the market. A build with real users beats it.

Talk to people in the role. Search LinkedIn for “AI product manager,” filter to People, and send:

Hi, this is who I am. I’m learning AI product management and I find your profile inspiring. Could I take 10 minutes of your time? I’m happy to compensate you for it.

Most will help. Most will turn the money down.

Ankit’s estimate for this step: one to four months.

Step 2 - Reach out

Five job boards, not one. In India, that’s Instahyre, Naukri, Indeed, Wellfound and LinkedIn.

And attach your portfolio to every application.

“People don’t owe you trust. You have to make them trust you.”

Two things I’d add:

1. Mirror the listing’s words. The ATS, LinkedIn’s Job Match and a ten-second recruiter skim all match on the literal string.

2. Put the build everywhere. Headline, About, Featured. Of 20 profiles I reviewed for my portfolio guide, 15 wasted Featured.

Step 3 - Build for them first

This is the step that actually gets replies.

1. Pick 25 to 30 midsize companies you admire

2. For each one, ask: if I were the PM here, what would I do in my first six months?

3. Build a deck or a prototype that answers it

4. Send it to at least two decision makers. Apollo and RocketReach both have free credits

5. Follow up three times in a week before you call it

Expect silence from the first four or five. Ankit’s blunt about why: your work isn’t good enough yet.

By the fifth, sixth, seventh, it is. That’s when replies start. And when you walk into the interview, you end up talking about your projects instead of hypotheticals.

Then Ace the Interview

There are at least 3 interview rounds you must be ready to master. Let’s decode each:

1. Fit tests risk

Can they trust you in their domain? Switching industries no longer takes years. A few months of projects in the new domain get you most of the way to credible.

2. Product sense tests judgment

The same judgment we talked about in section 1. Builder skills get you in the room. This round decides if you stay.

3. AI depth tests whether you built your projects

Nobody asks about the best-case scenario. They ask what can go off track, and only the person who built it knows. Built a RAG system? Ankit says expect: where does it break, what happens at 100 million documents, what if you swap the model, how did you evaluate it?

For the AI depth round, my evals episode with Aparna Dhinakaran is the best prep I know. My AI PM interview guide covers the rest.

When you rehearse, the best tool is my Job Search OS.

Just remember to practice out loud. Ankit was specific on that.

And that’s all for today. See you in the next episode :)

Get Transcript

Where to find Ankit Shukla

• LinkedIn

• X

• HelloPM

• HelloPM on YouTube

• HelloPM on Substack

PS. Please subscribe on YouTube and follow on Apple and Spotify. It helps!

PS2. I used AI’s assistance to write today’s piece. But I stand by every word.

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

• Customer.io - Send smarter messages using your product data

• Kameleoon - Leading AI experimentation platform

• Bolt - Ship AI-powered products 10x faster

• Jira Product Discovery - Plan with purpose, ship with confidence

• Product Faculty - Get $150 off their #1 AI PM Certification via code AAKASH150

Today’s Episode

On October 6, 2025, OpenAI launched its Agent Builder. By the next day, the posts were out: ChatGPT had killed n8n and Zapier.

It turned out to be one of n8n’s best weeks ever.

Now it’s Claude Code’s turn. Last month, this went viral:

So do you still need n8n?

That’s today’s episode with Jan Oberhauser, the founder and CEO of n8n. We talked about where n8n makes sense, where his own team uses Claude Code, and how to become an AI PM at n8n:

I’m giving a free webinar on the fundamentals of AI that anyone getting into PM must know. Tokenization, how LLM models work, all of it. Join me.

Newsletter Version

Here’s the newsletter version of our conversation:

1. What n8n does that Claude Code can’t

2. How to use n8n and Claude Code in tandem

3. Roadmap to becoming the AI PM n8n would hire

Subscribe now

1. What n8n does that Claude Code can’t

We have been called dead probably a thousand times. I don’t remember half of the things that killed us over the years. We’re still here.

—Jan

Over the course of 2025, Claude Code exploded. By the holidays, it surpassed n8n in search interest and never looked back:

This April, Anthropic also shipped Routines, which run Claude Code on a schedule in the cloud.

Is there really any space left for n8n?

I posed this question to Jan. From his point of view, things have never better.

In May, SAP invested at a $5.2B valuation, more than double the valuation n8n held the prior October. It also embedded n8n into Joule Studio, its agent builder. So every SAP customer (it has millions) gets n8n in the product with no setup required.

And today n8n has:

• 200K+ GitHub stars

• 1.5M active users

• 1,200 customers on the enterprise product

So how is n8n succeeding despite Claude Code? n8n evolved its use cases.

A year ago, people came to n8n for personal stuff, like an email summary. Now the growth is in business-critical workflows.

Claude Code is an agentic tool that runs Anthropic’s models in your terminal. n8n is an orchestration layer that connects your tools, models, and data on a visual canvas.

If it’s for when you cannot rely on something working 95% of the time. You want to be 100% sure that it really works. So, you choose n8n.

Jan gave us a course on how to build in it:

Step 1 - Describe it

Back in 2025, the viral n8n posts were all workflows. You don’t start on the canvas anymore.

Jan prompted his n8n AI Assistant to create an agent for a personal project. Like a typical Claude Code experience, it asked clarifying questions and came back with the full workflow.

Step 2 - Gate the risky tools

Your editing really starts on the canvas now.

The main thing you want to do is figure out your where to do approval.

For instance, Jan asked the agent to book a meeting with his VP of Sales for 3pm the next day. He’s added human approval for meetings.

So instead of book on its own, it showed him the event it was about to create and asked if it was allowed to. He clicked Add event. Then it booked.

Step 3 - Audit the run

Then it’s time to run and audit.

The delightful thing about n8n is every execution is logged node by node, with the exact input and output at each step.

So let’s say you add a deterministic node between the agent and the final output, the log will show which path it took.

And you can rerun the rest of the workflow from that exact point with the same data.

Step 4 - Hand-off

Then this is the special part of n8n: hand-offs.

How do you give something in Claude Code to someone on your team? You have to use Github, and it’s messy.

In n8n, you just invite them. They see version history and who changed what. Then they can publish versions with notes. You can also export a workflow and just email it.

So now you should understand the difference between Claude Code and n8n. Claude Code stops at step 1. n8n is the infrastructure to do steps 2-4.

2. How to Use Claude Code and n8n in tandem

One of the. most fascinating insights from Jan was that n8n are heavy users of Claude Code themselves!

So it’s worth figuring out where they use Claude Code and where they use n8n.

The rule of thumb is:

So, to know which workflows to move, I put together this questionnaire. If you get 2 or more yeses, move it to n8n.

n8n and Claude Code are very different products, and in the end you need both.

Claude Routines are built for one person running tasks against their own repos. Meanwhile, n8n is built for processes a whole team shares and signs off on.

You also don’t have to rebuild anything by hand when you move a workflow over!

Use this prompt:

3. Roadmap to becoming the AI PM n8n would hire

Having Jan on the podcast, I had to ask him about the AI PM job search.

Here’s what I learned about how they run product and what they hire for.

n8n’s product team runs in squads consisting of 3-5 engineers, 1-2 PMs, and a designer. They hire people for:

• Technical depth

  The more technical, the better, for every role. Even n8n’s designers are expected to use Claude Code.

• Builders

  Tinkerers who run home automation for fun, because they’re close to the users.

• Depth past the demo

  How an agent scales, how you evaluate it, and how you keep it reliable.

This gives you a really good path. Study technical interviews, build synthetic experience, and learn AI tools.

Here’s how I’d build a portfolio that passes that test:

1. Build your first n8n workflow with Pawel Huryn’s guide.

2. Then follow Mahesh Yadav’s learning path in How to Become a Builder PM. Prototype your next agent there, and promote it into n8n through the MCP server.

3. Write 20 test cases and run them as an eval.

As for the interview, n8n runs live sessions where you work through a problem together. Here’s my guide.

And that’s it for today!

Get Transcript

Where to find Jan Oberhauser

• LinkedIn

• X

• n8n

PS. Please subscribe on YouTube and follow on Apple and Spotify. It helps!

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by

Arize - Ship AI agents and features faster with fewer regressions.

Get a full year of Arize’s paid plan, along with 8 other AI tools, at bundle.aakashg.com.

Today’s episode

When Zapier published its AI fluency rubric, it went massively viral:

Tons of companies are following in their footsteps, explicitly adding AI fluency as one of the areas they evaluate their employees (and PMs) on.

So how do you get the transformative rating?

That’s today’s episode with Wade himself, the CEO and founder of Zapier. I present him 3 PM work products, and he grades them live:

Newsletter Version

Thank you for having me in your inbox! I also put together this newsletter guide to AI fluency for PMs:

1. The history of AI fluency

2. What capable, adoptive, and transformative look like

3. Why judgment still beats AI usage in the end

4. How to build your personal agent stack

5. Roadmap to a Transformative rating

Subscribe now

1. The history of AI fluency

AI fluency is only about three years old. Looking back at the history is helpful.

March 2023

On March 14, 2023 GPT-4 launched and the world changed. Remember, GPT-3 really first came out in May 2020. GPT-4 was a nearly 3-year wait. And it proved to be worth it.

GPT-4 was a huge leap that really proved that AI agents could be useful at work. Zapier themselves declared a company-wide code red and ran a week-long hackathon.

Weekly AI use went from about 10% of employees to about 50% in one week. By March 2026, Zapier put adoption at 100%.

April 2025

On April 7, 2025, Tobi Lütke posted Shopify's internal AI memo on X himself.

It had three main points:

1. The expectation is you reflexively reach for AI.

2. You have to show why AI couldn’t do the job before asking for headcount.

3. AI use will explicitly go into your performance reviews.

The memo became a genre within months.

At Microsoft, a senior engineering leader told her org that using AI is no longer optional, and that managers should factor it into evaluations.

Meta made AI-driven impact a core expectation in performance reviews starting in 2026.

Duolingo published a version of the same thing.

May 2025

Zapier had a different take. Instead of a memo, they released v1 of their AI Fluency rubric.

It went massively viral.

March 2026

But V1 had a problem, in Wade’s words:

“V1 only screened for usage. The floor is constantly moving with AI.”

So on March 31, 2026, Zapier published v2, and that’s what we’ll focus on:

V2 grades people on mindset, strategy, and building. It also added a new dimension: accountability. The more capable AI gets, the more damage low-accountability use can do.

As Wade said:

With AI, you can delegate the work, but not the accountability.

April 2026

Then, the memos began to crack.

Duolingo dropped AI usage as a formal review metric. Their CEO’s explanation:

I’m not going to force you to do that.

So by this summer, companies had tried two approaches. Mandate usage, or define what good looks like.

Zapier’s version is the one that seems to be lasting.

2. What capable, adoptive, and transformative look like

I did the work for this episode. I built 3 sets of artifacts and had Wade grade them live (no collusion).

They mapped to the three rating levels.

What Capable Looks Like

Capable for a PM is a ChatGPT assisted PRD in a pretty good format:

He pointed out it missed two things:

1. A prototype: Now that it’s so easy with AI, he would expect one.

2. Clear evidence: Where did customers ask for this? Gong calls, Zendesk tickets, Reddit threads, posts on X. AI makes sifting through that mountain of data fast, so it should be included.

Won’t you look at that! A CEO bringing up two of the key skills I’ve been banging the drum about by himself. The old PRD is dead.

Finally, as he said:

If there are things I know that you don’t know, that’s probably not a great signal.

So that’s a ‘Capable’ Rating. Now let’s look at ‘Adoptive.’

What Adoptive Looks Like

Here, I presented him a working prototype alongside a PRD that cited evidence. It was generated with my PM Operating System:

I managed to predict both his notes before the podcast!

What Transformative Looks Like

For the final rating, I showed Wade this:

It’s a full operating model for a product team where:

• Skills hold the team’s recurring work.

• Subagents review in parallel, including a skeptic that argues nothing should ship.

• The agent reads Zendesk, Gong, Amplitude and Linear directly.

• Everything the team learns lands in a git-versioned memory.

And on top of all that is the auto-build cycle:

It clusters raw customer signals, tags each as an observation, interpretation, or hypothesis, and flags anything contradicting a past bet. The PM decides what to promote. Then an auto-build chain produces a PRD, a review panel, a prototype, evals, and draft code, with a second human gate before merge.

Wade called it transformative:

This is not how product and software was built three years ago. Not even close.

3. Why judgment still beats AI usage

Dan Hockenmaier has this wonderful graphic, and we all should remember it:

We want to use AI with good judgment, so we seem like a Turbo Brain.

You don’t want to be labelled a Slop Cannon. That’s a good route to diminish your reputation with stakeholders and lose influence over time.

Say how much effort went in

The worst version is passing slop off as finished work. As Wade said:

“Now you’re making me do the task.”

I see this on my own team. Someone sends me a doc they didn’t filter, and what they’re really asking is “is this good?” That’s handing the judgment job up to me.

At Zapier, people say upfront how much effort went in. Wade will write something like “I’ve used AI to draft this and done a quick skim,” or the opposite, “I stand by every statement.”

Speed is fine. Just label it.

4. How to build your personal agent stack

Wade had a massively viral tweet that Zapier now has “more AI agents than employees.”

So I had to ask him about it. He showed off his “robot staff.” You can copy it. It’s the basic scaffolding you need to get capable or adoptive:

Agent 1 - The morning brief

At 6am, an automation pulls his calendar and to-do list and posts a brief to Slack. It’s the agent he tells everyone to build first, because connecting tools for the first time is easy here.

Agent 2 - The scribe

This is his favorite. At 5pm it loops over unanswered email, messages saved for later, and every meeting transcript from the day. It surfaces what he owes people and drafts the follow-ups.

Here’s a starter prompt for your own version.

Agent 3 - The agenda setter

Wade’s exec and board agendas are now mostly generated by an agent that reads his week of meetings, Slack, email, AI chat sessions and metrics. He approves or rejects each item. If a conversion rate dropped, the agent puts it on the agenda whether or not anyone wanted to raise it.

Agent 4 - The CEO CRM

Every Saturday night, an agent scans Zapier’s enterprise accounts for upcoming renewals and usage swings. By Sunday morning, 10 to 20 draft emails are sitting in his inbox, each with a reason to reach out this week.

How do you build these?

You can just point Cursor at this article with the Zapier MCP.

Wade doesn’t write the code himself. He describes the automations to Cursor and deploys the result to Zapier.

5. Roadmap to a Transformative rating

First, ask, do I really want a transformative rating?

You see it’s not that 20-30% of the team should have a transformative rating. Wade says just like 1 PM on a team of 40 PMs should get it:

That’s because the floor moved. What was transformative in 2024 is just adoptive in 2026.

On top of that, he said being Transformative all the time is a warning sign. It usually means you’re tweaking your systems instead of shipping. As Wade put it,

“If you are, you’re not actually getting work done.”

I asked Wade for a real example of a PM who got one.

Here’s what he shared:

Late last year, when Opus-4.6 landed, people at Zapier were building second brains: markdown files with their goals and projects, with a coding agent pointed at them.

One product leader decided a personal brain was too small, and built V1 of a company brain that every product person could use. In Wade’s words:

“Every product person at Zapier kind of got a jetpack put on their back.”

And no surprise, it’s something I’ve covered in the podcast :)

• Claude Code

• Hermes + OpenClaw

So really, being Transformative in many ways means implementing what you read here. And then making sure it means other people at your company work differently because of what you built:

1. Start by shipping your first pull request.

2. Then learn loops to automate your repetitive tasks.

3. Give your system a memory on Claude Code or Hermes or OpenClaw.

4. Then version everything you build for PMs on GitHub.

5. You can also bring the whole org on the same page with an all-in-one team OS.

6. Once the system is running, build a company OS.

Get Trasncript

Where to find Wade Online

• LinkedIn

• X

• Zapier

PS. Please subscribe on YouTube and follow on Apple and Spotify. It helps!

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by

Arize - Ship AI agents and features faster with fewer regressions.

Get a full year of Arize’s paid plan, along with 8 other AI tools, at bundle.aakashg.com.

Today’s Episode

Some product teams are AI-pilled. Others take ‘AI-pilled’ to a whole new level.

Together AI is one of those teams. They have to be. They are valued at $8.3B, building a cloud for AI companies. Many of their customers already are agents.

So after experimenting at the absolute frontier of AI, what comes out is a product team stack that is 4 key components:

1. A shared context repo any harness can read (not just Claude Code)

2. Skills for feature research, PRD writing, and prototypes

3. An orchestrator for leaders like their CPO

4. Agent evals

I got their CPO, Charles Zedlewski, two PMs, Necoline Hubner and Pavneet Ahluwalia, as well as their developer experience tool lead, Hassan El Mghari, to break down how everything works and was built.

If you want to see the full stack of an AI native product team, this is for you.

Thanks for having me in your inbox. I also wrote up my key takeaways for you across each of their four tools.

Subscribe now

1. A Shared Context Repo

Charles, their CPO, perfectly captured the problem with what using AI turns into for almost anyone in any team these days:

The new party foul is flooding your coworker’s context window.

So the key is to engineer a set of AI tools that don’t make an individual more productive, they make the collective more productive.

Two Key Components

This all begins with a shared context repo. Necoline, the architect behind theirs, walked us through it:

If you’ve seen the team OS I built, based on the one demoed by Hannah Stulberg, then the shape will be familiar:

• The context library is your home for shared context. Together splits theirs by product area as well as by mission/milestone in their strategy. Every team has access to the same library. So before Necoline pitches anything to a PM in another area, for instance, she reads that PM’s up to date context files and understands exactly what they care about.

• The skills folder holds all the shared workflows. That’s where they keep everything from PRD writer to user research. Necoline’s topline-status skill, for instance, triggers automatically at the end each sprint. It reads Linear, Github, and the strategy to draft a weekly leadership update of what is shipped vs ongoing vs up next. Instead of spending 30 mins chasing engineers, she just edits the AI output.

Four Governing Rules

It’s mostly markdown files, some YAML. Nothing fancy. What makes it work is 4 governing rules:

1. Every skill lives next to the thing it acts on.

A skill that only works against one codebase (deploy steps, API conventions) goes in that engineering repo with the code, so it stays in sync with it.

Skills that aren’t tied to any codebase, like research or PRD writing, go in your personal repo if they’re just yours, or the shared repo if everyone does that work.

2. Skills earn their way to main.

Every skill starts out as a personal skill. Then, if a PM is using it often, they will push it to the shared repo. But if it’s niche, it stays on a branch forever.

They don’t gate what skills exist in the repo by committee. It’s by usage.

3. Ownership follows the area

Each folder in the context folder is owned by the PM who owns the area. So because Necoline is the PM on Together’s Sandbox product, she keeps the Sandboxes folder current.

Then, at the higher layer, strategy files get rewritten at planning sessions or when the CEO sends a direction letter.

Beyond these two mechanisms, if someone pulls context and finds it missing, they update it.

4. Not married to Claude

This is a team that hosts models for a living. So they’re acutely aware of the costs, and their point of view is Anthropic has gotten expensive.

Instead, they prefer picking the model per task, using GLM for coding and prototypes, Kimi for analysis, and even their own models for some tasks. So they’ve built the repo to run on OpenCode, Hermes, any harness.

2. Skills for feature research, PRD writing, and prototyping

Two of the most important skills sitting in that shared team repo are feature research and PRD writing. Together’s PM for Infrastructure, Pavneet, walked me through the process.

It starts with Research

He started with the idea that customers wanted to resize shared storage. Instead of jumping straight into a PRD skill, he first fired off their feature-research skill.

The skill examined their support tickets, engineering tickets, internal docs and came back with a complete report:

Here’s what Pavneet had to say about the quality of this report (someone who could actually judge it):

If I had to do this manually, this would have easily occupied half of my day, if not more. This is actually pretty good.

That’s real time savings at quality. Then he clicked on one insight to get directly into their customer insights tool, where he could see a customer who wanted a self-serve option for more memory.

He built a fairly complete picture of the customer/user insights side, which would have taken him half a day when he was at Amazon, in five minutes.

Then you move into writing

After the research, Pavneet started their prd-writer skill. It read the research, shared context, then interviewed him turn by turn.

They’ve specifically instructed it to challenge the PM: tradeoffs, one way doors, and the like.

This is their solution to launching slop at your teammates. Their skill doesn’t just output. It focuses on getting the right stuff out of you.

Then you build the prototype

The PRD used to be the primary PM artifact. It was an elaborate gating document. Companies used it to make sure all stakeholders were aligned before you build.

When Pavneet worked at Amazon, it was a 20 page document with detailed PRFAQ. The Together team has replaced that with one to two pages plus a prototype.

Those pages cover:

• The customer problem, defined well enough so everyone groks it

• Solution options, not necessarily fully worked out

• A sample user journey for the leading solution

The rest of a PM’s effort goes to using the ux-prototype skill, which creates a detailed prompt for Figma Make.

The prototype is where the real debate amongst stakeholders happens, because now everyone can click through the solution instead of imagining it.

While everything was human in the loop till now, after alignment on the prototype, it’s mostly automated. Build and ship is automated, kicked off with a single /goal that fans out sub agents and comes back with a PR. Then engineers review it.

3. An Orchestrator for Leaders Like Their CPO

PMs like Pavneet and Necoline have different use cases for AI than leaders like their CPO Charles. So he showed me where he spends his time:

What it does is it has every repo for every product in one place. Charles picks a repo, picks a harness (OpenCode, OpenCode Claude, or Cursor), picks a model, and asks a question.

He gave the live example of model shaping, their product for adapting open weight models. He asked which model was most recently enabled for supervised fine tuning. They have 30+ and the list changes every week.

Orchestrator spun up a sandbox, cloned just the slice of the repo it needed, and came back with the answer:

The Orchestrator inherits every skill and MCP server that lives locally in each team repo, server side. So Charles gets the latest tooling for any codebase without installing any of it.

As Charles said:

Most people don’t have a whole lot of motivation to want to understand all the depth and nuance of the context of somebody else’s area. They want to know just the amount they need to get one single question answered.

That’s where the Orchestrator comes in. Hassan, the developer behind it, said it took a few weeks to a month of engineering to build.

4. Agent Evals

Charles told me something that reframed the whole tool for me:

At this point, for a lot of our products, agents are already the majority user.

So when Together ships a feature, the test isn’t a user study. It’s whether Claude Code, handed their docs and an API key, can actually complete the task. Hassan built the tool that runs that test, and showed me:

Hassan sits with each PM and writes tasks that represent real usage. One from fine tuning is to run a job on this dataset, deploy the result as a dedicated endpoint, hit it with inference, evaluate the output, then delete the endpoint. The tool spins up a sandbox, launches Claude Code with that prompt and a key, watches it work, and grades it.

Then it runs the same task three ways. Docs only, with the MCP server, and with skills. So they can see which context layer is actually doing the work. It’s the eval discipline Daniel McKinnon showed me, pointed at your own product instead of a model.

The part that matters is the improvements tab:

On this run the agent couldn’t figure out which models were fine tunable. Hassan opened the transcript. The page existed. It just wasn’t linked from the fine tuning quick start, so the agent never got there. One docs PR fixed it. Dozens of docs fixes have come out of this exact loop.

Compare that to how you used to validate a design. Weeks of user tests, then conflicting signal. Now agents re-verify every few hours that the product is what the docs say it is. Charles calls that the new bar for UX.

If your users are humans on a GUI, you’d swap in a visual reasoning model that clicks through screens. Same architecture, different grader.

Bottomline

Where does the product manager end and the developer begin?

All in all, the essence of each job hasn’t changed. A PM brings unique and well-validated customer problems. An engineer brings designs that most efficiently meet their needs. Same as before AI.

What changed is reach. A PM can now finish small and medium tasks inside engineering. An engineer can open Customer Insights and run a query on what their customers need.

So how will you stand out as the edges are blurring?

I predict that PM roles will become product builder roles in the coming years. Then your differentiating factor will not be an individual AI setup, no matter how suave. Everybody will have one.

Your leverage is giving your whole org a bird’s-eye view. Start small today. Build a personal OS, expand to your team, then your company.

Try ‘em out and let me know how it goes! And I’ll see you in the next one :)

Get Transcript

Where to find the Together AI Team

• Together AI

• Charles Zedlewski

• Necoline Hubner

• Pavneet Singh Ahluwalia

• Hassan El Mghari

PS. Please subscribe on YouTube and follow on Apple and Spotify. It helps!

Check out the conversation on Apple, Spotify and YouTube.

Brought to you by:

1. Customer.io - Send smarter messages using your product data

2. Ariso - Ship AI agents and features faster with fewer regressions

3. Viktor - AI employee connected to 3k+ tools. Every action approved by your team

4. Bolt - build a complete design system from your code

5. Product Faculty - Get $150 off their #1 AI PM Certification: code AAKASH150

Today’s episode

2023: Prompt Engineering
2024: Context Engineering
2025: Harness Engineering
2026: Loop Engineering

As the founder of OpenClaw said:

Loops are all the hype now. I’ve already written the web’s deepest guide to loops for PMs. Today, I bring you the practical screenshare.

My guest is , Chief AI and Product Officer at JobNimbus, which just raised Utah’s largest Series B ever.

And they are operating in the future. Design is building design loops that meet product loops that PMs build that meet engineering loops that engineers build.

Today, Tyler shows you everything:

Today’s deep dive

As a thank you for having me in your inbox, I wrote up my key takeaways and expanded upon some of Tyler’s key insights. Here’s your guide to loops for PMs:

1. What makes a loop a loop?

2. How to write your first loop in 10 minutes

3. 5 Loops every PM should build

Subscribe now

1. What makes a loop a loop?

The first thing to understand is why do we care about loops? Tyler put it very well:

Where prompting got hard is you always had to keep yourself in the loop, so you didn’t feel the gains from what you get from Agentic.

So that’s why we have loops. What is a loop?

Anyone can easily say ‘build a loop in claude code’ or run an agent forever and call it a loop. But are those legit? Probably not.

Tyler visualizes a legit loop like this:

1. Fetches its own inputs

2. Does the work

3. Has a gate

4. Writes a named artifact

And goes back to fetching its own input again.

That’s basically it. An agent or prompt become a loop when you feed the session back in and let it rewrite itself.

2. How to write your first loop in 10 minutes

Here’s some real alpha for you: write your first loop by hand, not with AI.

The advice genuinely surprised me, but Tyler was firm on it:

I’ve read some papers that suggest that skills authored by humans are often better than the ones authored by AI. You know more and can express more of what you want this thing to do.

And then he gave a second reason, too:

People are like, yeah, I got an ebike, but don’t worry, I still want to get exercise and we’ll pedal. Once you get on the ebike, you’re not pedaling.

Once you start using AI to write loops, it’s like an ebike. You only use the AI for pedaling (writing the loops).

So he opened an empty file and typed a skill by hand live. This is roughly it:

# Decide

Help me make good decisions.

I struggle with decision fatigue. Push me toward a decision once enough
thinking has been done.

Use contrarian thinking. Question me, use the Socratic method, and call
my BS. Bring in the latest research where it's relevant.

Be succinct.

That’s a skill. Then you save it, test it, and improve it. When that’s done, it’s one easy command to make it a loop:

Can you set this up as a loop and ask me what you need including a learning hook before closing out sessions that learns from the log.

It’s as simple as that. You set up a skill. You test it. You improve it. Then, you set it up as a loop.

You can also close the loop by hand, which is worth doing once so you see what it's actually doing:

Based on this conversation, how should we improve our skill to be better and more effective next time? 

It works in regular Claude Code. Tyler uses herdr, a terminal tool for running several Claude agents side by side, which further simplifies the process because it will set them off with the keyword ‘agent’ in your Claude Code prompts:

3. 5 Loops every PM should build

Tyler recommends PMs build at least these 5 loops:

Loop 1 - Customer outreach

This is the one Tyler says he gets the most value out of himself:

It’s a really good way to keep close to the customer without one of the biggest challenges, which was just the time to go find who to reach out to and send them a message.

Loop 2 - Project status

This is the boring one that saves the most hours. Pull the tickets, find what’s stale, find who’s blocked. Nothing clever about it.

Loop 3 - Thinking partner

This is the opposite of what everyone’s building. So I want to highlight it.

Tyler’s team writes skills that push back on the person instead of doing the person’s work:

It’s less about doing and more about pushing the thinking of the people on the team. Did you talk to customers? Did you look at the benchmarks in the industry? Where does the data come from?

The reason is that the treadmill is genuinely addictive:

You get all this stuff back and you’re like, I can’t possibly go and evaluate this, so I might as well just send it up the chain and see what happens. That’s not doing product work. That’s outsourcing your thinking.

That’s career damage for a PM in 2026. A year ago a huge deck made you look like a machine. Now people recognize it on sight and know you didn’t think.

Loop 4 - Divergent prototypes

While we were talking, his product skill was building prototypes in the background off a prompt this thin:

Create a crew schedule prototype for a contractor in a product like JobNimbus.
Build three variants: A minimal, B full-featured, C creative.

And this led to 3 divergent prototypes in just a few minutes. From Tyler’s point of view, you should always do this:

Never skip it, in my opinion. We used to talk about what your test coverage is, and now it should just be 100%. It’s basically free. Why not do it?

And then his product loops actually uses a synthetic customer to review the prototypes before any human sees them.

Loop 5 - Quality Watchdog

This is one to ask engineering for, not build yourself.

Here’s the math Tyler walked me through, and it’s the thing most teams are getting wrong right now:

If you ship twice as fast with AI and your quality rate maintains the same rate, your customer experiences twice as many bugs.

That’s his read on why uptime at big companies has been so bad lately. So build this loop to avoid low uptime!

And with these 5 loops, you’ll be ahead of most PMs out there.

Final words

A large part of the PM job used to be production for the majority of the last decade. That work was slow and deliberate and hence the most valued. It is now close to free.

So where does that leave you? Loops. My prediction is that they will be the fastest way to hand over the production half of your job to something that never gets tired of it.

And with the hours won back you can focus on calibration and discovery. So build the loops and then go spend the time in a room with a customer.

I will see you in the next issue.

Get Transcript

Where to find Tyler

• LinkedIn

• X

• YouTube

• Website

• AI Architect

Go Deeper

If you don’t know where to start on Claude loops, I got you covered with my ultimate guide on Loops for PMs. This includes my library of 12 top PM loops.

Then follow it up with The Complete PM Guide to /goal in Claude Code.

Finally, Tyler mentioned the evolving role of PMs and how it is shifting into a Product Builder role, so you can learn more in my podcast with Mahesh Yadav on How to Become a Builder PM.

Finally, please subscribe on YouTube and follow on Apple and Spotify. I have much more content like this coming!

Share Product Growth

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Bolt.new - Ship AI-powered products 10x faster

2. Product Faculty - Get $150 off their #1 AI PM Certification: code AAKASH150

3. Customer.io - Send smarter messages using your product data

4. Land PM Job - 12-week live course to master the PM job search

5. Viktor - Use $100 in starting credits to get 5x more done with this AI employee

Today’s Episode

I’ve showed you how to build a PM OS, team OS, and company OS in Claude Code. Today I show you how to build a Company OS on Hermes and OpenClaw.

Because Hermes and OpenClaw get you:

• Something that lives on Slack and Telegram

• Something that runs 24/7

• + Self-improvement

These are unique features that really take your Company OS to the next level. And if you’re the PM to lead the project, they give you tremendous impact.

Mikhail Shcheglov is the CPO at OLX Uzbekistan, and he’s had one in production for the last 5 months, so he’s showing you how to build one, mistakes he made along the way, and what it can do for you.

Enjoy:

Today’s Deep Dive

I’ve written up the key takeaways for newsletter subscribers as well:

1. Where to grab the open sourced version

2. How to create your own Company OS with this

3. What the future of product development and management look like in this world

1. Where to grab the open sourced version

Mikhail open-sourced the structural skeleton of the agent that runs his working life at OLX.

Go grab it for yourself:

Saul Goodman

The main caveat is: it’s still a skeleton. You’ll have to supply the vital organs, i.e., context, to watch it come to life.

Let’s show you how.

2. How to create your own Company OS

Now it’s time to really understand the system you’re building better. We’re using a blend of two frameworks:

• OpenClaw is the base runtime because it comes with everything out of the box: Slack and Telegram gateways, scheduler, persistent identity

• Hermes drives automated skill generation and self-improvement. It watches which tasks you request most often and builds them as skills

These come together in 4 system layers, with 3 layers of memory.

As the graphic shows, there are 5 steps to build it. Let’s cover each:

Step 1 - Measure First

Instead of jumping into the scaffolding, in classic PM fashion, let’s start with a metric. It’s called product context coverage, and here’s what it looks like when Mikhail maps it:

Pretty cool, huh? Here’s a prompt you can use to generate it. You can steal it:

The visualization is a separate two-liner afterward. Mikhail one-shotted his in Fable:

Once you have this dashboard, you now have something to goal on.

Mikhail’s screenshotted agent currently reads 54%. At 54%, the Company OS operates like a capable junior-to-mid PM that can make backlog decisions. Once you get to 70-90%, it can actually start doing strategy-level work.

Mikhail keeps that as a personal OKR.

The graph doubles as a management tool. You can drill into any team to see how much context they hold. Silos show up on sight.

For instance, his clusters revealed barely any overlap between the buyer and seller experience product teams:

That means those teams aren’t talking, and a leader can help create that connective tissue.

Step 2 - Scaffolding

The next step is to set up the scaffolding with Claude Code. Once you have forked the repo from Section 1, you’ll see the whole anatomy:

Now we need to fill it in. Hit it with this prompt:

It’ll quiz you as you asked, and then you’ll have everything built out:

Step 3 - Wire Memory

Now we’re ready for memory. Memory has 3 layers. Let’s build them bottom up:

Hit it with a prompt like this:

This will get your memory wired up.

Step 4 - Imperatives

Now let’s talk imperatives. Here’s the underlying problem, in Mikhail’s words:

LLMs are focused on the resemblance of good output rather than the actual results.

The fix is a long, tested list of “imperatives.” You can organize them like this:

So hit it with this prompt:

Step 5 - Put in Repo

Now you’ve done all the setup work, and it’s time to publish. Read my guide to GitHub if this is unfamiliar to you.

Otherwise, hit it with the prompt:

Since every change now lives in GitHub, you can tweak the agent from the Claude app on your phone, the way Mikhail runs his.

Then begin testing with your team!

3. What it means for your job

Here are some use cases you can test this on:

1. Stakeholder gatekeeper - ask stakeholders to ping Saul before a PM

2. Board skill - ask Hermes to abstract each board member’s mental model and give feedback on your next strategy

3. Chief of staff - ask it to give you a morning digest of your email and calendar

4. Recruiting - ask it to build a pipeline for outreach

What you’ll see once this is up and running is…

We’re slowly realizing with AI that the job of product manager is more important than ever.

You can build infinite tools like this, and they’re even more effective on the coding side, but you can’t replace true discovery, coordinating with stakeholders, true strategy, and the accountability to make sure things work as they were planned.

That’s where a PM lives, and that’s your wedge.

Use this to automate all the other work.

Get the Transcript

Get More of Mikhail

• X

• LinkedIn

• Corporate Waters

• Saul the AI Chief of Staff

Go deeper

Start with my guides to Hermes and OpenClaw. Then learn how to build Memory and a Company OS.

You might also enjoy my collaboration last year with Mikhail on What PM Interviewers Look For.

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Check out the conversation on Apple, Spotify and YouTube.

Brought to you by

1. Customer.io - Send smarter messages using your product data

2. Ariso - The AI operating partner for every manager and team

3. Product Faculty - Get $150 off their #1 AI Builder Fellowship. Code AAKASH150

4. Land PM Job - Join me for Cohort 5 starting November 16th

5. Amplitude - Custom agents that monitor your funnel and file the ticket

Today’s Episode

Freshworks is a $3.4B giant of SaaS. They’ve been around since 2010. They have over 75,000 customers and 4,000 employees. Their 2026 revenue will be $960M.

They’re a colossus.

When Srini Raghavan joined them as Chief Product Officer, they had a 6 month release process. Under his tenure, they moved to a 2 week release cycle.

They embraced a new way of working powered by AI.

In today’s episode, he breaks down everything:

1. The AI PDLC they embraced

2. The AI Harness they created in Cursor, so it can run on any models

3. How the PM role shifted to Product Builder

I hope you enjoy it as much as I did:

I’ve written up the key takeaways for newsletter subscribers as well.

1. The AI PDLC they Embraced

How do you go from shipping every 6 months to every 2 weeks?

It’s not about becoming AI first! Most teams try that: they open Cursor or Figma Make and start prompting.

Srini made the point that it actually all begins with being Data First:

When you’re data first, you create the right foundation to actually be AI first. At Freshworks scale, with 300 million end users, you can’t afford a hallucination.

For Freshworks, that was three things:

1. A design system structured so agents can parse it

2. Coding standards written down explicitly

3. A single repo as the source of truth

They wrapped that all into a system they call Prism. It’s a knowledge hub that knows the product and its dependencies inside out, plus a context hub that passes feature context between phases, and a central library of what they call AI builder artifacts. These are the skills, rules, commands, and agents that describe how Freshworks specifically builds.

The AI PDLC sits atop all of that. And it looks like this:

It’s the same lifecycle every SaaS company runs from discovery through release, with two changes. First, there’s a governed AI agent working inside each. Second, there’s an evals phase at the end.

And that’s how their release cycle went from 6 months to 2 weeks.

But even in the episode, there were some cracks. Figma Make skipped a few design system components. Srini noted that those are the places where humans still have a role.

2. The AI Harness in Cursor they created

I began in engineering. After 14 years of not touching code, now I’m spending a lot of time in Cursor.

Srini is now spending lots of time in Cursor, and he showed us. Everything starts with a slash command, /fw-innit. Here’s what that kicks off:

The harness asks for the business unit, Epic ID, and feature team. Then it runs 12 phases, all the way from idea brief through to prototyping and QA.

Srini did the whole demo on Grok models! He explained:

I used to use Claude, but Grok seems to be working really well, and it’s really really fast. Nothing takes more than 10-15 seconds.

And indeed, that’s what we found. If you really need speed, consider Grok.

He live built a performance dashboard for their new EX Agent Studio. After he described the feature in plain English, the agent came back with questions a good PM would ask:

• How deep should the drill down go?

• Who is the primary persona?

• What does success look like in 6 months?

It then wrote its own SQL against Bel, their Databricks data lake with usage from all 75,000 customers. It assembled this into quantitative evidence in the PRD:

It looked across all 4,358 active ITSM accounts and showed its work.

At the end of the process is a step called CPO check which reviews the draft the way Srini would. His team named it after him.

From there, the PRD goes straight into Figma Make. Because of being data-first, they have a design system built into it. And after a few prompts we had a working prototype:

That’s the end-to-end process. No Claude Code or Codex. Just Cursor + any model.

3. How the PM role shifted to product builder

Which brings us back to where we started: what roles will execute work like this in the picture?

Srini manages 200+ people with titles like PM, UX researcher, and designer. His prediction is:

He didn’t say that to get clicks on Social Media. That’s how he’s running Freshworks. When he joined, Freshworks ran 1 PM and 1 designer for every 10 to 20 engineers.

Now, their new workflow is 1 PM to 1 engineer. Some teams have no dedicated designer at all.

Here’s why: those titles marked stations on an assembly line. The PM would write the PRD, hand it to design, and design would hand it to engineering. Now, like in section 2, one person can do the whole assembly line.

If you want to prepare for this future, I’ve put together the roadmap:

Pay close attention to step 8. Since one person can now run this assembly line, that person also owns whether the thing made any money.

Get more of Srini

Freshworks is hiring and you can find Srini on LinkedIn. He also generously shared his deck.

Since the episode was recorded, Srini actually left Freshworks. With his tenure as CPO at RingCentral, SVP at Five9, and Director at Cisco, I’d expect you see him at another high-powered role soon.

Go Deeper

• Hear from more CPOs on becoming AI native: Rachel Wolan, Jiaona Zhang

• Learn more about becoming a Builder PM: Mahesh Yadav

• Explore other Coding Harnesses: Codex, Claude Code

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Bolt.new - Ship AI-powered products 10x faster

2. Product Faculty - Get $150 off their #1 AI PM Certification: code AAKASH150

3. Customer.io - Send smarter messages using your product data

4. Ariso - Ship AI agents and features faster, with fewer regressions

5. Land PM Job - 12-week live course to master the PM job search

Today’s Episode

Whether you’re using a PM OS, or using regular Claude, or using another AI harness (like ChatGPT Work or Codex)…

Today’s guest Oji Udezue, CPO at Typeform, Calendly, and Parsable, and he goes into a masterclass of what skills PMs should have, how to build them, and how to use them.

Enjoy:

Apple Podcasts

Spotify Podcasts

The landscape of PM automations is changing rapidly. Between graphs, loops, scheduled tasks, cowork, claude code, lindy, relay… it’s hard to break it all down.

I’m giving a free webinar on Claude Automations next Tuesday. Join me:

Join the Webinar

Do you want to get 1:1s with me to talk AI, PM, and landing a job? If so, join the next edition of my Land PM Job cohort program:

Join the Cohort

Today’s Deep Dive

Thanks for having me in your inbox. Here’s your hand-written key takeaways:

1. Why PMs need a broad range of skills

2. The top skills every PM should have and use

Subscribe now

1. Why PMs need a broad range of skills

Oji is currently running ProductMind, where he works with companies to become AI-native. And one thing he has observed is:

We see PMs who are not speeding up their product judgment, speeding up their orchestration skills to match the new speed of the engineers

There are loads of repos out there for tokenmaxxing, or engineering, but there are almost no good repos for PM skills (one exception: mine).

The way Oji conceptualizes a business in his book Building Rocketships, every company has three layers:

1. Business

2. Product

3. Code

Only having skills at the bottom most layer is a huge gap. That’s why Oji has generously open-sourced business and product skills:

Install Library

2. The top skills every PM should have and use

Oji’s system has 17 skill files:

The most important thing about these skills is what they cover. They cover the stages of product development that actually require PM judgment: finding the problem, deciding what to build, getting people to value, making money, setting direction, and running the work.

In addition, there are two unique things about how Oji set up his skill system:

1. The project scaffolding skill

2. The concept of gates

Let’s walk through each.

The Project Scaffolding Skill

The way Oji has orchestrated the system is it has a new project scaffolding skill at the heart of it all.

What executes is an 11-step workflow, and the first 4 steps never touch code.

By the end you have a market research doc, a product brief, a project-specific CLAUDE.md, a folder structure, a tests folder, CI wired up, and secrets already in .gitignore.

The Concept of Gates

LLMs very rarely tell you no. And this skill will tell you absolutely no, don’t do this.

Everything in Oji’s skills is built around this fact.

Take vet-a-feature. It’s built on the idea of the sharp problem test:

Other skills similarly work with gates. And what this does is it helps you build trust in the LLM’s response and replicate real PM judgment.

You want to build skills with gates that tell you no.

Final Words

With AI where it is today, the scarce skill is no longer building something. It’s the judgment to say this one and not that one. I call it Taste at Speed.

You cannot prompt your way to this skill, and you can’t hire an agent to hold it for you. You have to own it as a PM.

As much as these automations help…

Get the Transcript

Get More of Oji

• LinkedIn

• ProductMind

• ProductMind Substack

• The open-source skills repo

• Shipyard

• Building Rocketships

Go Deeper

I’ve done extensive testing on what makes a good skill and how to build a PM OS for paid newsletter subscribers (that go much deeper than this podcast).

→ Get the founding plan to get access.

I also have several free podcasts on Claude Code that you may find helpful: my 3 part series with Carl Vellotti, PM OS with Dave Killeen, Team OS with Hannah Stullberg, Company OS with Jiaona Zhang, and Claude Code for CPOs.

→ To never miss an episode, subscribe on YouTube and follow on Apple & Spotify.

Finally, in my live course, I teach you how to do extremely advanced PM automation Claude Code.

→ Join us.

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. SerpApi - Get started with SerpApi using 250 free credits.

2. Product Faculty - Get $550 off their #1 AI PM Cert with code AAKASH550C7

3. Ariso - Ship AI agents and features faster, with fewer regressions

4. Land PM Job - 12-week experience to master getting a PM job

5. Pendo - The #1 software experience management platform

Today’s Episode

Rasty Turek spent the past year building with coding agents, and he mapped how his process changed over that time.

His eval started as QA. Then it became the spec. He reckons he now spends around 90% of his time on evals. And the difference is “indescribable.”

Just about every developer and builder has experienced a similar pattern.

Just as the value of a long PRD was waning, evals sprang up. And PMs are often considered the best owner.

So how do you do them well?

I’ve now covered evals basics, excel implementation, advanced theory, Braintrust implementation, and Arize implementation.

Today, I cover how to practically write a great one to hand off to another team.

And I do it with the perfect person.

I’ve brought in Daniel McKinnon, who was a PM on the Llama models at Meta. He wrote the viral piece “Show, Don’t Tell” where he said:

When a partner team wants Llama to do something, I ask them to please not send me a product spec or PRD or description of the problem. This just adds additional work for the team to parse and understand. Cut the middleman and send me an eval directly.

So what would a good eval to send him look like? Today, we show you.

Apple Podcasts

Spotify Podcasts

🆓 I’m doing a free webinar Thursday on getting AI PM interviews. Join me:

Join the FREE Webinar

The next cohort of my LandPMJob program starts August 17th! This is cohort 4, and will be the best yet. Sign up.

Today’s Deep Dive

This is the guide to building your first eval before you have a single user.

1. Why your first eval works differently

2. How to build your eval set in Claude or ChatGPT

3. How to score it

4. How to read the number

5. How evals can help your career

Subscribe now

This post was written entirely by hand:

I use AI in my editing and ideation phases, but am here to give you HUMAN content.

1. The cold-start problem

Everything I’ve taught you about evals so far has the same first step: Look at your data.

That’s the error analysis method, and it’s still the most calibrated way to build evals over time. But it has a prerequisite. You need outputs to analyze.

This is the gap Daniel’s method fills. He builds the eval before the product exists, out of domain knowledge instead of production data. I’m calling it the cold-start eval, and it’s the one most PMs never learn.

When a product is expected to do nearly everything, a normal PRD usually folds because it goes into specifics like how it behaves in particular situations and how the user gets value.

So examples become the specification.

“It’s really just like a trivia question for the model.”

Offline vs Online Evals: An Explainer

Say you run a recipe site and you want to generate ice cream recipes.

The bet you’re making is that a good offline score predicts a satisfied online user. It doesn’t always hold, yet it’s still the best predictor available.

Sometimes the offline eval tells you the thing you want to build is not possible with today’s models, and you find that out in week one instead of after launch.

Then you make changes accordingly.

One note before we build, Daniel ran his live demo in Codex. The method has nothing to do with the tool, so I’ve written every step below in a way that could work in Claude Code or ChatGPT Work also.

2. Step-by-step, how to build your eval set

You have a feature idea and no data. So this is how you get to a working eval set in one sitting. The shape of what you’ll build will kinda look like this ↓

Step 1 - Write the problem in one sentence

You have to understand exactly what the feature will do in one sentence. For example:

Step 2 - Validate your domain expertise

You will not write a good eval for a space you don’t know. So, if you’ve never shipped an AI feature, Daniel’s advice is blunt.

And that’s where I come in :D

Step 3 - Set up the workspace

Open Claude or ChatGPT and create a Project. Upload whatever your feature reads, like, sample support tickets, example transcripts, a product spec, your scoring guidelines. Then paste this into the project instructions:

Step 4 - Find your floor

Pick the easiest genuine case you can think of and check the model can do it at all. Daniel picked cystic fibrosis - a disease with a known genetic cause and a canonical gene behind it.

While Daniel’s version was pretty niche, your version can look like:

Now run it on the model you’ll use. Call this the floor test. It’s where you find out that your easy case sits above the ceiling of the model. If your floor fails, you can make the case easier.

Step 5 - Find your ceiling

Now the opposite move. Find a case you’re fairly sure nothing can solve. But why bother testing something you expect to fail? Because if you can’t find a case that fails, your eval is already saturated and it will never tell you anything again.

Step 6 - Fill in the middle

You have a floor and a ceiling. Everything else is a binary search between them.

Run that a few times and keep going until you have a set. How many? Around 100 is typical. And yes, you use AI to build the test for the AI. That’s the only way this takes 90 minutes instead of two weeks.

3. How to score it?

Daniel described the scoring step but didn’t write one out on the recording, so I built a version for you all. It’s built on the binary pass/fail principle from the eval guide.

Calibrate the judge before you trust it. Do this once, at the start. It would take around 30 minutes, but it will make a huge difference between a score and a guess.

4. How to read the number

Say your eval comes back at 50%. Is that shippable? Probably not.

So you go one level down and ask which slices you’re good at. Then you put guardrails on the product so it only answers the kinds of questions it gets right 80% of the time.

That’s a PM call. Everything below the line goes back to the research or engineering team with a specific ask. You hand them a target instead of a feeling.

5. How evals can help your career

Daniel estimates there are probably fewer than 100 PMs building frontier models worldwide.

And, as he said:

The average PM is a motivator and an analyst, and all of that is getting easy to do with AI.

Evals, on the other hand, are purely built on judgment. So, this is the place to go deep and differentiate. As you will be one of the few who can.

P.S. Many of you are looking for a great AI builder role. Daniel’s hiring.

Here’s a handy infographic summarizing what we covered:

Share

Where to find Daniel McKinnon

• X

• LinkedIn

• Gamow Labs

• His excellent blog

Go Deeper on Evals

1. AI Evals for PMs: Everything You Need to Know to Get Started in 2026

2. How to Run Evals in Claude Code with Aparna Dhinakaran

3. AI Evals with Hamel Husain and Shreya Shankar

4. Evals are the New PRD with Ankur Goyal

5. AI PM’s Guide to LLM Judges

6. AI Evals Explained Simply with Ankit Shukla

To never miss an episode, subscribe to the podcast on YouTube and follow on Apple & Spotify

Last Few Things

I’m doing a survey on PM salaries. Please fill out this 3-minute survey. I’m giving away 10 copies of Eric Ries “Incorruptible” to participants.

• To get access to my AI tool stack - become an annual subscriber ($150), and grab Aakash’s bundle.

• To get access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

And, as I mentioned earlier, my PM cohort starts next month with a free Webinar Thursday.

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by: Arize

Trace, evaluate, and fix your AI agents before broken behavior ships to users.

Improve your agents

Today’s Episode

Eric Ries is the Lean Startup Guy. He was an early call from Dario Amodei when he was founding Anthropic.

Dario could afford that call and make the connection. But what about you?

I’m making that call for you today. Eric spent the last few years creating his next movement after Lean Startup, Incorruptible, and I get the low down:

• How to structure your company with a trust

• Governance structures you can copy

• And the underlying culture you need

He goes behind the scenes to show how he wrote Incorruptible with his AI tool Solveit and live screenshares his writing workflow with Claude.

Check it out:

Apple Podcasts

Spotify Podcasts

🎊 To celebrate, I’m giving away 10 copies of Eric’s book Incorruptible:

All you have to do is fill out this short survey on PM salaries.

🆓 I’m doing a free webinar Thursday on getting AI PM interviews. Join me:

Join the FREE Webinar

The next cohort of my LandPMJob program starts August. If you want my 1:1 coaching, sign up.

Today’s Deep Dive

I’ve written up the PM’s Guide to Governance:

1. The four dimensions of Incorruptibility

2. The Costco case study to become Incorruptible

3. How to structure your next company (if you’re a founder)

Plus, two bonus sections:

1. [Bonus 1] How Eric Ries Uses AI to write

2. [Bonus 2] The Lean Startup methodology in the AI era

1. The four dimensions of Incorruptibility

If you start studying the world’s most lasting companies, the first thing you’ll learn is that:

Most spectacular startup collapses were governance failures, not product failures.

So how do you avoid them? Four things:

2. The Costco Case Study

Let’s talk about Costco for a second. It’s one of the best performing companies of all time:

How has Costco had such longevity? Well, it actually starts with disappointment. You see, Costco was co-founded by a guy named Sol Price.

Sol originally created FedMart. It became a very successful company, but he gave control to investors. Eventually, the board wanted growth while Sol would do things like tell customers to go buy from a competitor down the street if they had a cheaper price.

So, the board locked Sol out of his own company. It didn’t work well for them. By 1982, FedMart was in liquidation.

Sol, and his mentee Jim Sinegal, didn’t want to follow in FedMart’s footsteps. So when they built Costco (a merger of their two companies), they built a governance fortress, with staggered board, poison pill, fiduciary hierarchy, and a cultural fortress:

The cultural fortress is perhaps the hardest to capture part. How do you build an Incorruptible culture like Costco?

Eric takes inspiration from Jim Sinegal:

Compromising is "literally like taking heroin," one small price increase, stock goes up, you get hooked. Now it's baked into the forecast, so you have to do it again next quarter.

You have to avoid “financial gravity.”

3. How to structure a company from day one

Dario Amodei called Eric when Anthropic was still figuring this out.

Here’s the advice you can get from that call - without the access:

Seed Stage

1. Convert to a Public Benefit Corporation. Write the corporate mission directly into the charter. It is a legal structure that gives directors permission to pursue purpose even when it conflicts with shareholder value.

2. Designate a mission guardian. Someone with the authority and responsibility to keep the company on course. At Answer.AI, the founders hold that role.

Series A-B

Like Anthropic created and defended a Long-Term Benefit Trust, you will have to do the same.

Investors may want to change the terms. Hold the line.

Series C+

At larger companies, a mission lock vehicle can take the role of mission guardian:

This is what Answer.AI, Eric’s company did, and what Anthropic did.

Just remember, structure isn’t everything.

So that covers it on our short tour through governance for PMs. Now, two bonuses.

Bonus 1: How Eric Ries uses AI to write

The thing to avoid in the AI era is being called slop. How does Eric do it?

The book Incorruptible was written inside Solve It, the AI platform from Answer.AI. The approach is fundamentally different from prompting.

How it works:

1. Three block types. Notes (raw markdown you write), prompts (LLM instructions), and code (Python). You mix and match freely.

2. Context control. The LLM only sees messages above the current one. You choose what it knows.

3. Direct editing. If the model writes something wrong, instead of saying “try again,” you replace what it said with what it should have said. The model learns from the corrected version, reducing the mistake-correction cycle.

4. The writing style guide. A document loaded into every session listing phrases to avoid, patterns to follow, and the author’s specific voice characteristics. The generic AI gets banned before it starts.

Eric then took it to the next level. He had 600+ test readers leave structured comments, which the tool helped him organize and do months of daily turns on.

Bonus 2: The Lean Startup methodology in the AI era

No. But you have to understand what it actually is.

Build, Measure, Learn is a principle for operating under uncertainty. Its essence is:

• Learn as quickly as possible

• Turn every iteration into feedback

• Shorten the time between building, measuring, and learning

• Repeat faster than everyone else

The framework has been used on power plants, cancer therapies, even the long-term stock exchange. It works in the AI era too.

What matters is whether you can go faster than the convention in your industry.

If everyone else in your space takes 18 months between major iterations and you take 12, you have the advantage.

Claude Code is emerging as a standard partly because Anthropic’s product organization has unparalleled velocity.

Shipping velocity does more than accelerate learning. It compounds trust through two effects:

1. The cooking effect. When a company is shipping consistently, getting better visibly, you start to feel safe building on top of their platform. Your app will magically get better over time because their platform keeps improving.

2. The standardization effect. Once enough developers choose a platform, everyone else has an incentive to do the same. Enterprise customers sign multi-year contracts. Partners build around it. Hiring shifts toward it. Over time, the platform becomes the default choice.

And how does Claude Code ship so fast?

And that’s how we come full circle.

Get Transcript

Finally, here’s a one-sheeter you can print on governence:

Where to find Eric Ries

• LinkedIn

• Incorruptible

• Answer.AI / Solve It

• Long-Term Stock Exchange

• Lean Startup

PS1. If you want access to my AI tool stack - Airtable, Dovetail, Arize, Linear, Descript, Reforge Build, Magic Patterns, Speechify, and Mobbin - grab Aakash’s bundle.

PS2. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by: Hyperagent

Turn your recurring PM work into reusable agents now.

Start here

Today’s episode

If you’ve got an enterprise Claude subscription, you have access to one of the most powerful tools ever invented.

But at the same time, many PMs report using AI but not feeling any more productive.

Here to help is Jyothi Nookula. She just used Claude Code to beat 30 teams at her company’s hackathon. She’s getting productivity out of Claude that’s measurable.

And today she breaks down everything.

I’ve done plenty of advanced guides on PM OS, Team OS, and company OS. I’ve also broken down Claude Code for non-technical PMs and the Claude ecosystem.

Today’s piece is the first principles, bottoms-up explanation of everything Claude:

Spotify

Apple Podcasts

Want to Become an AI PM?

Let me help get you there live. The upcoming cohort of the Land PM Job program includes a full AI PM course:

Join Us

Newsletter Deep Dive

As a thank you for having me in your inbox, here is the full workflow, aligned to the five layers of the Claude Ecosystem.

1. Models - when to choose which

2. Surfaces - when to use which

3. Knowledge - how to build it out

4. Integrations - how to manage them

5. Agents - the one’s to build

   • How to build an AI Chief of Staff

Subscribe now

The 5 Layers of the Claude Ecosystem

Claude has a dizzying array of features and surfaces. You can think of the Claude ecosystem as a stack of 5 layers:

1. Models at the bottom. These are the raw intelligence.

2. Surfaces sit on top of models. These are things like the the web app, desktop app, mobile app, or Chrome extension you actually type into.

3. Knowledge sits above that. This is your projects, skills, memory, and custom instructions.

4. Integrations comes next. Your MCP connections to Slack, Drive, and Jira; plus, your own files.

5. Agents and orchestration sit at the top.

You want to move up the layers one by one.

Layer 1 - Choosing the right model for the job

This is the first skill: knowing when should you which of the three models.

Here’s what Jyothi recommends:

She walked through each in detail:

Haiku is your speed machine. Cheapest, fastest, best when you need volume over depth. Triaging a pile of docs, tagging, quick classification, generating fifty variants of something. In the episode, Jyothi runs her scheduled automations on Haiku specifically to save tokens.

Sonnet is where ninety percent of PM work lives. The best ratio of quality to cost. PRDs, user research synthesis, competitive analysis, stakeholder briefs, roadmap thinking. Start here for almost everything.

Fable/Opus for the high-stakes reasoning. Genuine trade-off analysis, contradictory research, long-horizon planning where you need second and third order implications.

Layer 2 - Picking the right surface for the task

So you know the model. Now, what surface? Here’s what Jyothi recommends:

• Chat on the web is your thinking-out-loud surface.

  Quick questions, a smarter replacement for a Google search.

• Desktop is where the real work runs, because it can touch your files, your connected systems, and your scheduled tasks.

  Cowork and Claude Code both live here.

• Mobile is for kicking off a run and checking on it from a walk.

• Chrome Plugins is for computer use.

  Competitive research, pulling from sites agents cannot otherwise reach, and user testing where you tell Claude to behave like a real person and report back where your product confused it.

• Claude Design for decks and prototypes.

Upload your brand guide, a Figma file, or a GitHub link, and everything it generates matches your company’s colors instead of looking like a generic template.

It also has a real visual editor built in. You can mark up a slide, drop a comment the way you would in Figma, and Claude executes the edit (as shown in the above image).

In the episode, Jyothi mentioned that she built a CEO-level deck in the hour before a meeting that looked like it took her all day. For a quick design, Claude Design is enough.

• Claude Code when you want a clickable app that real users can poke at and give you feedback on.

Layer 3 - Build out your knowledge base

This is the layer Jyothi says PMs underinvest in most, and I agree.

It’s what turns Claude from a generic chatbot into something that actually knows your world.

We need to cover three pieces: projects, skills, and memory.

Projects are your folders

You want one project per role. A project should hold all your context through memory and custom instructions.

If you want to run three different things at once, make three projects, each carrying its own context as project instructions.

Skills beat prompts

A prompt is a one-time instruction. A skill is a saved playbook Claude reaches for on its own.

Jyothi’s customer interview synthesis skill is the template to steal. It has a when-to-use trigger, a checklist, and numbered steps: inventory the inputs, extract observations with citations so it hallucinates less, use the speaker’s own words, and keep behavioral observations separate from stated preferences.

Since early this year, skills can be multiple files. If your main skill file is one giant markdown doc pushing past 500 lines, you’re doing it wrong.

Two more things Jyothi flagged:

1. Write your skills yourself. Her read of the research: AI-written skill files underperform human-written ones. Draft with Claude, then layer in your domain knowledge, your template, your structure.

2. Update on drift. Review skills when your domain changes or when output quality slips. Declining output is your cue that the playbook went stale. Quarterly is a fine default.

The highest-value skills to build first: prioritization, PRD writing, customer interview synthesis, and turning support tickets into Jira issues.

Memory is the compounding piece

If you’re using projects in Claude web, then tell the project all of your context and tell it to commit to memory:

Here’s my context. Commit this to the project’s memory:

1. My role:

2. My company:

3. Our strategy (attached doc)

4. Our OKR attainment last quarter (attached doc)

5. The big features we’re going after:

If you’re using Claude Code, grab this memory system I built for you.

It captures as you work, with every claim traced to its source and conflict markers.

Layer 4 - Integrations

MCP is how Claude connects to the systems your work lives in. Slack, Drive, Jira, Gmail, your CRM, your analytics.

Two types:

1. Remote MCPs are the ones you click to connect. Customize, connectors, hit the plus, authenticate. Done.

2. Local MCPs run on your machine. This is how Jyothi’s chief of staff knowledge base works. We’ll build it below.

Which ones do you need? Look at the tools you already touch daily. Email, calendar, Slack, meeting transcripts, your Jira board, your analytics.

Jyothi’s most exotic connection at work: NVIDIA’s BioNeMo model for drug prediction.

Layer 5 - Build out agents to do things for you

There are three parts to agents that you want to build:

1. Automations that kill your busywork

2. Self-improving agent loops

3. A chief of staff

Let’s talk about each.

Agent Category 1 - Automations that kill your busywork

Start with Cowork. You need the desktop app and at least a Pro plan, around twenty+ dollars a month, and you can schedule Claude to run on its own.

Jyothi recommends a few standing automations:

• A morning brief at 9am wired to Calendar, Gmail, Drive, and Jira. It pulls the day’s meetings with attendees and attached docs, the inbox threads that need her, and the Jira items on her plate, then hands her the top three things to focus on.

• An end-of-day wrap at 5pm that reads the morning brief, checks what actually happened, and previews tomorrow. What shipped/missed, what is new.

• A standup brief on the active sprint. Done since yesterday, in progress, blocked, new, and sprint health, rendered as a clean dashboard.

Two things to know before you build.

These automations only run when your laptop is on, so time them for when you are actually working.

And this quietly replaces the Relay, Lindy, Gumloop, and n8n flows where one failed node kills the chain.

Agent Category 2 - Self-improving Agents

This is the ceiling, and it is where Jyothi’s hackathon win came from.

She went up against 30 engineering teams and won with one idea from an Anthropic blog post (Adversarial agents).

The idea comes from an older AI technique called GANs. One AI creates something. A second AI tries to find everything that's wrong with it. The feedback goes back to the first AI, which improves its work. The cycle repeats until the output is good enough.

Now apply the same idea to agents. One agent builds, another looks for flaws, and the first agent keeps improving until it passes your quality bar:

The promp to kick it off looks like this:

Agent Category 3 - Your AI Chief of Staff

Connect Claude to your sources and it can read your calendar and act on your Jira. What it still does not have is a model of your world.

It does not know who your allies are, which relationships are load bearing, or which message is too sensitive to send before you loop someone in.

A chief of staff closes that gap.

Jyothi built a personal agent that reads every meeting transcript and learns her org over time.

After one meeting, chief of staff told her to make a particular colleague an ally because they were strong in an area she was trying to grow into.

Another time it stopped her before she sent something and told her who to inform first. That’s not possible with just connectors, you need a knowledge graph of your working life.

How to Build It

The shape is straightforward:

• A context KB, structured into folders for people, meetings, documents, company, your priorities, and the patterns it spots over time.

  

• A set of extraction templates so a person profile captures how they operate and what they care about, and a document gets its metadata, key points, and relevance to you.

• Then an MCP server on top, so you can talk to the whole thing from the desktop app instead of digging through files.

Feed it so it compounds

The system is only as good as what you put in it. Rank your inputs by signal density and feed the richest first.

• Meeting transcripts are the richest source. Pushback, reactions, who went quiet, what got decided. Wire an automation so every new Granola or Google Meet transcript that lands in your inbox auto-logs to the KB.

• Key strategy docs you write and receive.

• Slack threads, where a surprising amount of real context lives.

There is a fair objection here - Log every email and every transcript and you will burn tokens.

That’s true.

The payoff is a knowledge base rich enough to connect dots you would have missed, and you can throttle it to only the sources that carry real signal.

Get Transcript

[Bonus] Takeaway

Here’s the things to remember to use Claude better. Print it out:

Share

Where to find Jyothi Nookula

• LinkedIn

• NextGen Product Manager

Go Deeper

1. To get access to my AI tool stack - Airtable, Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

2. To get access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Refer a friend

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by: Arize

Trace, evaluate, and fix your AI agents before broken behavior ships to users.

Improve your agents

Today’s Episode

Meng To is one of the world’s leading AI design thought leaders, with 170K+ followers on X and 16K+ on Instagram. He told me something wild:

I barely use Claude anymore.

What does he find is more powerful? Codex. So today’s episode is a masterclass in designing with Codex.

I’ve covered the Codex harness, PM use cases, and design basics already. Today’s episode goes the next layer deeper:

I hope you enjoy the episode as much as I did.

Apple Podcast

Spotify

Want My Coaching?

You should know the story of Sourav Yadav from my Land a PM Job course. He closed 5 offers in 2 months and became Head of Product at Times of India.

I’m starting my next cohort next month. We’re teaching three courses:

1. AI PM ($3000 value)

2. PM fundamentals ($2000 value)

3. How to get a PM Job ($6000 value)

Here’s the syllabus:

This cohort features two 1:1s with me (first time ever).

So, what’s stopping you?

Apply now

Want a taste of the course? I’m putting on a free webinar on behavioral & AI PM interview mastery next Thursday. Sign up.

Newsletter Deep Dive

Now, thanks for having. me in your inbox. Here’s a complete guide to AI Design for PMs.

1. How to Setup Codex for AI Design

2. The AI Design workflow that works

Subscribe now

If you don’t have time for the full deep dive, here’s the most important takeaways.

Share

1. How to Set up Codex for Great Design

Codex is ChatGPT on Steroids

What makes Codex ChatGPT on steroids? Three things:

a. Context
b. Skills
c. Connectors

If you cover these correctly, you get a much more powerful harness for AI. So let’s briefly cover each.

1a. You need to build out great context

If you’re using my PM OS, you know this already. If you don’t, here’s what you need to do:

We recommend 1 folder per project, grouped under a single parent directory:

This works because you don’t have to re-explain yourself to AI all the time: what you do, what your company is going after, what your team’s product strategy is.

It all loads in for free.

Now here’s the trick: it’s a Goldilocks problem.

Too much undifferentiated context and the AI burns tokens figuring out what matters before it even starts the task you asked for.

Too little context and you spend forever iterating on the output when a bit more upfront context could’ve one-shotted what you want.

So get really good at managing that context. Even ask Codex:

1b. You need to enable it with the right skills

I’ve been talking about skills a lot. So you probably already know they are a lightweight MD of instructions. And you should have them for everything.

To design AI that’s not slop, you should know about this taste skill that has 60K+ stars on Github:

Also install the Playwright skill to take screenshots of UI for GPT to iterate on itself.

1c. You need to connect it to the right tools

You’ve heard about MCPs for the last year. Use all of the MCPs! PostHog, Linear, etc.

Designing for AI with Codex has two nuances when it comes to connectors:

1. You want to use plugins (not MCPs) to connect to Figma and Slack.

2. You also want to install ChatGPT Atlas so Codex can use computer use on websites for tasks that don’t have an API (eg browse LinkedIn).

How these three features come together

Context + Skills + Tools suddenly transforms your LLM into an agent harness that is much more powerful.

And two new Codex features make this more powerful than ever:

1. Codex Mobile sits inside your ChatGPT app and connects to whatever is running on your desktop, so you can check in on a build from a taxi or a coffee line.

2. The screenshot shortcut (cmd + cmd) lets you select any window and drop it straight into your current chat as visual context, no separate upload step required.

2. The AI Design Workflow to Avoid Slop

So now that you have the setup, here’s the workflow:

1. Plan with context

2. Build with the taste skill

3. Iterate with a human touch

It sounds simple but the power is in the nuance.

Step 1 - Plan with context

You’ve probably heard of plan mode. Use it!

The fastest way to lose an afternoon with Codex is to skip straight to building, since it commits to a direction within seconds and rarely doubles back on its own.

Plan mode forces a pause between your request and the AI’s first action.

Read that plan the way you’d read a junior PM’s spec before a kickoff. If something is missing, ask before approving. “Can we expand on this part” or “what if we add this constraint” costs you thirty seconds.

Give it the right context

Screenshots are the highest leverage form of context you can give the AI, more useful than a paragraph of typed description.

Two things are worth knowing here.

1. Simply select a window, hit the screenshot shortcut (Press both Command keys at the same time), and the image drops straight into your chat as the thing to fix or match.

2. If you have an app on different devices, run the app in a simulator, capture every screen in the flow, sign in, payment, editor, and hand the AI the full set so it can validate the design end to end instead of one screen at a time.

A screenshot tells the AI what you mean in one shot. A paragraph of description tells it what you think you mean, and those are not always the same thing.

Step 2 - Invoke the taste skill

The model matters less than the taste layer you give it. Use the taste skill from 1b and then look at the outputs.

Usually the skill will give you several - soft, brutalist, etc. If it doesn’t for whatever reason, prompt it:

Once you have a design you like, the next decision is output format:

• HTML is the fastest and most controlled option, the default unless you have a specific reason not to.

• Figma takes longer and assumes Figma fluency, but it’s the right call when a designer needs to pick up your file and keep iterating.

• Keynote or PowerPoint only makes sense when the final deliverable has to literally be a slide file someone opens in that app.

The rule of thumb is to keep your blast radius small. Every extra tool you add is a subscription, a login, and a context switch the AI cannot do for you.

Step 3 - Steer it mid-flow and iterate

One of the coolest features of Codex vs ChatGPT is once the build is running, you are not locked out!

You can steer mid-task, queue a follow up command while the current one finishes, or interrupt with new direction the moment you see it heading somewhere you didn’t intend.

So do that. And then iterate more on the AI output. The difference between slop and tasteful design is typically found in this final 20% of iterations.

And that’s it - now you’ve got design that isn’t slop!

Get Transcript

Downloadables to Take Away

1. Taste Skill - The Anti Slop Frontend Framework for Al Agents (60k+ stars)

2. The 15 tools running Meng To’s Al Workflow

   

Follow Meng Online

• Design+Code

• LinkedIn

• YouTube

• Aura

• X

Go Deeper

Tools:

1. To get access to my AI tool stack - Airtable, Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

2. To get access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

Podcasts:

1. How to Design with AI - YouTube | Spotify | Apple

2. How to Use Codex Like an OpenAI PM - YouTube | Spotify | Apple

3. The Ultimate Guide to ChatGPT Codex - YouTube | Spotify | Apple

Newsletters:

1. OpenAI’s Codex is the Best Way to Use ChatGPT

2. I spent 100s of hours building a PM OS for you

3. How to build product strategy in the age of AI

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Refer a friend

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Ariso - Ship AI agents and features faster, with fewer regressions

2. Bolt - Ship AI-powered products 10x faster

3. Pendo - The #1 software experience management platform

4. Product Faculty - Get $550 off their #1 AI PM Certification: code AAKASH550C7

5. Customer.io - Send smarter messages using your product data

Today’s episode

I’ve written about building a PM OS and Team OS in Claude Code. They are my top two most popular pieces of 2026.

Today is the upgrade: a company OS that enables even CSMs at your company to ship to production.

And this isn’t theoretical.

Jiaona Zhang “JZ” is the CPO at Laurel - which raised a $100M Series C - and her company actually runs on a Company OS. They have run into the problems and kinks. They’ve ironed them out. Now, they’re showing what works.

JZ is not some “AI purist.” She has led product at Airbnb, Dropbox, Webflow, and WeWork. She’s an AI realist.

And in this episode, she doesn’t hold anything back:

• The Company OS that enables it all

• How PMs go from product idea to shipped feature with agents

• How she conducts AI PM interviews in this era, and how to ace them

This is one of the densest episodes I’ve ever recorded.

I hope you enjoy watching it as much as I did recording it.

Apple Podcast

Spotify

• To get access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

• To get access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

Newsletter Deep Dive

Welcome to the web’s first guide for how product leaders can create a Company OS to accelerate Claude Code adoption across their org:

1. How to build your Company OS

   • How to spread AI adoption beyond engineering

2. The new role of the PM in an AI-native team

3. How to get hired as an AI-native PM

Subscribe now

1. How to build your Company OS

A company OS can unlock productivity across your organization. It takes the power of a concept like a PM OS beyond single PMs or product teams.

At Laurel, even CSMs are shipping to production.

Here’s how to build your own:

The architecture is simpler than you think

One GitHub repo. Every team in the company has a folder. Every folder maps to the activities that team does. Every activity has a skill file. The skill files go directly into Claude’s organization settings. Anyone who opens Claude already has the right skill loaded.

During the demo, JZ screenshared Laurel’s full GitHub structure live. Customer success, data science, design, engineering, finance, legal, marketing, all of it:

There are three layers to build your own.

Layer 1 - Start with ontology

PMs build the OS and then wonder why nobody uses it. They got the order wrong.

Start with the ontology like JZ. Map every function’s work to categories, then to tasks within each category. What should sales be doing? What should CS be doing? What should product be doing? This work map is what informs the folder structure. The OS is built from the ontology:

At Laurel, the ontology sits alongside the OS. For each function, it shows which activities should go up (more human time) and which should go down (automated away). For product, competitive analysis write-ups, stakeholder briefs, and research synthesis are in the “stop doing manually” column. Feature work, QA, and direct customer contact are in the “spend more time here” column. Color-coded. Visible to every leader.

This is the part most companies skip. They install tools before deciding what work actually matters.

Layer 2 - The skill file architecture

Once you have the ontology, build the skills.

A skill file is a Markdown document that encodes how to do a specific piece of work. Not just a generic prompt. A specific, opinionated guide that reflects how your company does that thing. How do you write a renewal email at your company? What does a good feature request look like when it comes from a CS rep? What questions do you ask before triaging a support escalation?

The skill file answers all of this. Upload it to Claude’s organization settings and it becomes available to everyone. The skill is called when needed. With this simple set up now one person’s best workflow becomes everyone’s default.

Layer 3 - The delivery layer

At Laurel, every customer facing team member gets a daily morning message in Slack. Calendar, meetings, check-ins, onboarding sessions, all surfaced. And embedded alongside each item, the skill to use. You have a renewal call at 10am. Here is the renewal skill. You have an onboarding session at 2pm. Here is the onboarding skill. The OS meets people where they already are.

How to spread AI adoption beyond engineering

When AI adoption is everyone’s responsibility, it is no one’s responsibility. The person who was going to build the CS workflow is also running renewals, handling escalations, and onboarding three new accounts. The workflow never gets built. Leads to a huge gap.

The fix ↓

Step 1 - Dedicate a person to it

At Laurel, this role is called AI Operations. One person, full-time mandate, whose entire job is finding efficiencies, building workflows, and spreading what works to every function.

JZ describes it as the new BizOps:

AI ops is the new BizOps. Before, BizOps were doing really meaningful things, but often it was very high level, all the different hats, a lot of like market level stuff. Now, if you repurpose this idea of having BizOps, which is really again a Swiss Army knife in many ways, to finding people who are insanely curious, tinkering with the latest technology, and relentless about finding efficiencies, that’s the DNA I really look for.

Laurel started with one person, Sasha. Within months, every other function wanted their own Sasha. That demand is how you get budget. You demonstrate value with one person, then let the pull happen naturally.

If you are reading this as a leader who cannot yet hire for this role, the person closest to it already exists on your team. They are the one who figured out the Slack automation nobody asked them to build. Find them. Give them the mandate officially.

Step 2 - Run a companywide hackathon

The assumption that blocks AI adoption in non-technical functions is simple: you have to be technical to build something.

One hackathon breaks that assumption faster than any training program.

Laurel ran a companywide hackathon at their offsite roughly three months ago. Every team participated. Not just engineering. The explicit goal was to show that anyone can build. When a Customer Success manager ships something in a day, the story spreads. That story does more for adoption than any top down mandate.

If a full offsite feels out of reach, a single afternoon works too. Pick one function. Give them four hours and a clear problem to solve. Keep a low bar, make it like even a working Slack automation counts. A prompt template that saves 30 minutes a day counts. The goal is the first win over the perfect solution.

Step 3 - Make the culture visible

Some companies have a value on a slide that nobody reads after the presentation is done.

Laurel turned their “unreasonable hospitality” value into a workflow. When a Customer Success manager has a check-in coming up and has not had an in-person touch with that customer in a while, the OS surfaces it. Pulls from Gong transcripts what the customer loves. Suggests the gesture. Handles the logistics. If you can see, now the value is not just a slide anymore.

That is the part most leaders miss. Culture spreads through systems that make the right behavior the default behavior. You do not need everyone to be intrinsically motivated to show hospitality. You need a system that makes it easy enough that everyone does it anyway.

2. The new role of the PM in an AI-native team

There is a version of this conversation happening on Reddit right now. A PM asking if their job is going away. Getting 200 replies, half of them panicked and other dismissive. Tbh, neither group is right.

The job is not going away. The shape of it is changing faster than most people want to admit.

Here is what the new shape actually looks like.

The captain model

Forget the old handoff chain. PM writes spec, sends to designer, designer sends to engineer, engineer sends back, designer says that is not what they designed. That loop is going down in AI-native teams.

The replacement is a captain model. Every feature has one person who owns it end-to-end.

The captain is whoever has the most important skill for that feature’s hardest problem:

• Engineering captain - Architectural changes, system overhauls, anything where the codebase risk is the biggest variable.

• Design captain - Interaction heavy features where the experience is everything and the engineering is straightforward.

• PM captain - Features where customer understanding plus business context is the hardest thing to get right. Empty states. Onboarding flows. Anything where the content and the user insight matter more than the code.

That’s the full model.

Two track product reviews

Speed without alignment is just running in circles. The captain model needs guardrails.

At Laurel, there are two tracks:

Track 1 - Fast.

Small features, single captain, end-to-end ownership. Goes through the Ask Devin reviewers Slack channel. An engineer reviews the PR. A designer checks if needed. No formal product review meeting.

Track 2 - Full review.

Radical changes to core user interactions, architectural decisions, system level thinking required. This is what JZ calls a product strategy review. “Are we sure this is the right direction for the whole product?” Plus a separate architectural review - “Will the system actually support where we want to go?”

The question most teams get wrong is which track a feature belongs on.

The test is simple. If the change affects a core user interaction that touches the whole product, it is Track 2. If it is scoped to one workflow and the hardest problem is execution, it is Track 1. Temporary initiatives, despite being a frontend and backend feature, was Track 1. A redesign of how Laurel surfaces time across the entire day would be Track 2.

I covered the mechanics of PMs shipping to production in my PM guide to shipping your first pull request. The captain model is what makes that shipping sustainable at scale rather than a one-off experiment.

As an ending note, I would say…

3. How to Get Hired as an AI-Native PM

Ask someone on your team if they use AI. Everyone says yes. Ask them to show you. That is where the answer changes.

There is a framework that cuts through the noise immediately. Four levels. Every person, every team, every company sits somewhere on this scale.

Level 1 - Chat mode.

You open Claude or ChatGPT. You type a question to get a specific answer.

This is AI as a slightly smarter Google. Most people are here. If you are closing the tab after every session and starting fresh next time, you are Level 1.

Level 2 - Workflow automation.

You stop doing a task manually and build something to do it for you.

A Slack automation that triages feature requests. A template that auto-populates from your CRM. A morning briefing that pulls your calendar and surfaces priorities. One workflow. That is the entry point. If you’re not familiar, I covered how to start building these in my AI Agents for PMs guide.

Level 3 - Building apps.

You identify something tedious enough to deserve a real tool, and you build an app.

Something with a UI, logic, and state. This is where builder PMs live. Mahesh Yadav broke down exactly how to get here in our episode on becoming a builder PM.

Level 4 - Shared apps and shipping to customers.

You are building things other people use. Your Customer Success team ships a feature. Your PM submits a PR that goes to production. You are in the full product lifecycle. This is where AI-native teams operate.

In most organizations, the engineering team is split between Level 2 and Level 3. Sales and Customer Success are mostly Level 1. Finance and legal are often at Level 0, which is not even on the scale.

The screen-share test

Here is how JZ uses this framework in practice. In every interview, she asks the candidate to screen share.

Within 60 seconds you know exactly where someone is. A Level 1 person has a bunch of Claude chat tabs open with one-off questions. A Level 3 person has a folder structure, a set of saved workflows, maybe an app they built last week.

Here’s the most important takeaways in 1 infographic:

Get Transcript

Where to find Jiaona Zhang

• LinkedIn

• Reforge

• Laurel

Related content

Podcasts:

1. How a VP Uses Claude Without Producing Slop - YouTube | Spotify | Apple

2. How to Build a Team OS in Claude Code - YouTube | Spotify | Apple

3. How to Become a Builder PM - YouTube | Spotify | Apple

Newsletters:

1. I spent the last week building an OS in Claude Code

2. I spent 100s of hours building a PM OS for you

3. How to build product strategy in the age of AI

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Refer a friend

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by: LogRocket

This is the AI analytics, logs, and session replay tool you need. I ran a head-to-head eval against PostHog to test their AI, and published my results here.

Check it out

Today’s episode

There are hundreds of guides on writing PRDs with Claude. Dozens on running user interviews. Almost nothing on how a VP of product actually uses it in-depth.

Matt Wensing is the VP of Product and Design at Customer.io. They crossed $100M ARR, just shipped an AI agent, and are one of the fastest growing companies in B2B SaaS right now. I asked him to show me his actual documents, his actual Slack threads, and the exact sessions where Claude helped him produce leadership grade output.

What he showed me changed how I think about AI for leaders. Claude has the instincts of a brilliant new hire, it wants to deliver before it fully understands what you need, and at the VP level that gap shows up fast.

Matt has spent months figuring out how to manage it, and in this episode he shows you everything. You’re not going to find another video in the world with real VP of product examples from their real work.

Apple Podcast

Spotify

• To get access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

• To get access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

Newsletter Deep Dive

Thank you for having me in your inbox. Here is how a VP of product at a $100M ARR company actually uses Claude:

1. How a VP of Product Actually Works With Claude

   • Why Claude produces slop, and why executives catch it instantly

   • How to build leadership grade presentations with Claude

   • Using Claude as a thinking partner vs. output machine

2. A VP of Product’s AI stack

Subscribe now

Before we get to the longer article, here’s the most important takeaway in an infographic.

Share

Now into the deep dive…

1. How a VP of Product Actually Works With Claude

Why Claude produces slop, and why executives catch it instantly

Matt has a rule he shares with every new hire at Customer.io:

Junior employees, however talented, will race to the finish line before they have enough context to do the job well. They hear the first instruction, want to impress, and just go.

They skip the clarifying questions. They come back with something that looks complete but misses the point entirely, and now you have to tear it down and start over.

Claude does the exact same thing.

Give it a goal and it will sprint. It will generate a low grade Word doc. It will use jargon mid session and put it straight into the executive document. It will suggest the next step before you have finished thinking through the current one, nudging you toward a deliverable you are not ready for yet.

Matt calls this “the slop problem.” And it gets worse the higher up you go, because the audience reading your output gets sharper.

Executives are the best slop filters in the room

Senior leaders spend their careers reading documents that try to persuade them. They have a filter that kicks in within the first paragraph. They are not looking for polish. They are looking for evidence that the person who wrote this actually understands the multi dimensional nature of the problem.

A one shot alignment deck fails that test every time. Years of organizational history and unspoken context get compressed into a clean seven slide narrative. Matt calls this the “flat projection problem.” You present a shadow of the problem to people who have been staring at the real thing for months. They filter it out, and depending on where you sit in the hierarchy, that looks like polite nodding or your work simply getting ignored.

The fix is a different way of working with Claude entirely.

Instead of giving it the full problem upfront, you feed it context in layers, one piece at a time, making it earn the right to generate the final output. You kill the eager suggestions the moment they show up. You stay in control of the pace.

Matt puts it simply - a 200 iteration session with a great deliverable at the end is worth far more than saying yes to the first draft.

The document you produce is less polished but higher in insight density. When an executive reads something that matches the complexity they have been living with, they do not filter it out. They lean in.

How to build leadership grade presentations with Claude

In the episode, Matt shared an interesting story, he woke up at 5am with a company all hands due at 11. His week had been fully booked, the way every leadership week is (you know this better).

He started with - Six hours, a blank slide deck, and Claude.

By 11am, he had a polished Q2 roadmap presentation in front of 400 employees.

Here is exactly how he built it.

Step 1: Take inventory before you touch Claude

The single biggest mistake leaders make is jumping straight into building. Matt’s first move was not opening Claude. It was taking stock of what raw material he already had.

In this case he had two things. A Zoom recording of the engineering team’s demo day, and a strategy doc outlining Customer.io‘s three investment themes for the year. Neither was shaped correctly for the audience he needed to reach. The demo day was engineering talking to engineering. The strategy doc was internal direction, not a story for 400 people across sales, marketing, and engineering.

Raw ingredients first. Always.

Step 2: Pivot, don’t write

Once Matt had his ingredients, he did something specific. He did not ask Claude to write slides. He asked it to reorganize the demo day Zoom transcript around the three themes from the strategy doc.

Think of it as matrix multiplication. You have content shaped one way. You have a framework shaped another way. You ask Claude to pivot the first into the shape of the second. The output is not a finished product. It is strategically shaped raw material you can actually work with.

Step 3: Using Claude as Script Writer

Matt is a believer in show don’t tell. So he built the slides before he wrote a single word of the talk track.

Once the slides were done, he took screenshots of each finished slide and fed them back into the same Claude session. Then he asked Claude to write the talk track using all the context it had already ingested, the Zoom transcript, the strategy doc, the slides themselves, with one specific instruction. Do not just repeat what is on the slide. Use everything you know to say something more interesting than what is already there.

The result was a talk track that added depth instead of narrating the obvious.

Using Claude as a thinking partner vs. output machine

Matt does a lot of his Claude sessions by voice, on walks. You will see why this detail matters in a moment.

He was trying to build a pricing philosophy document for his CEO. The kind of document that shapes how an entire company thinks about value and growth for the next two years.

He did not tell Claude any of that upfront.

Start in the abstract, reveal the domain last

If he had opened with “help me build a pricing philosophy for Customer.io,” Claude would have pattern matched against every pricing philosophy document in its training data and produced something that looks like all of them. Generic tiers. Value metrics. A document that could have been written for any SaaS company.

So Matt tricked it.

He opened with a biology metaphor. An ecosystem. A two by two grid. Things enter from one corner, move through it, sometimes grow, sometimes stall, sometimes leave. He asked Claude to think about the different paths something could take through this system.

If you can see, he talked in completely abstract terms, never mentioning customers or pricing.

Claude had no idea where he was going. And that was exactly the point.

Here's the exact prompt he tried, transcribed straight from voice mode, stumbles and all:

Layer complexity in slowly

Matt did not dump everything into the session at once. He started with the simplest version of the framework, four boxes, basic rules of movement between them. Once Claude understood that, he added one layer of complexity. Then another and another.

He compares over prompting to a bad game night host. The person who explains every rule and every exception before anyone has touched a card. Nobody understands the goal. Nobody knows how to win.

Feed Claude the core rules first. Let it stabilize on those. Then add the exceptions.

Reveal the domain only when the model is clean

It was not until deep into the session, many iterations in, that Matt finally told Claude what the exercise was actually for. A pricing philosophy document for his CEO.

You have spent all that time building a clean mental model in the abstract. The moment you reveal the domain, you can apply that model as a stress test. Does it generate things that match reality? Does the framework hold up when pointed at actual customers? If yes, you have something worth presenting to a CEO.

2. Customer.io’s full weekly AI stack

Most of what you have read so far is about Claude desktop. Long sessions. Iterative context loading. Abstract frameworks that get applied to real problems. That is the deep work layer of Matt’s stack.

But there is a whole other layer running underneath it, every day, without Matt having to think about it.

The analysis bot

His team built an internal Slack bot with access to Snowflake. Matt had 2,000 customer records he needed to analyze for a report going to other executives. Instead of waiting for a data pull or writing SQL himself, he talked to the bot in natural language. Asked questions. Got answers. Followed up. Went deeper.

Two things he is careful about here. One, he has a data team that can jump in when the bot produces something that does not look right. Two, he never takes a non-deterministic answer at face value for something going into an executive report.

The Slack scanner

Customer.io has over 400 employees spread across dozens of Slack channels. There is no version of Matt’s job where he can read every thread that might need a product perspective.

So one of his teammates built a scanner. It runs across several dozen channels continuously, looking for conversations where a product person should probably be involved but is not. It surfaces those threads to Matt at set times during the day, without overwhelming him.

Matt is clear about what this tool is and is not. It keeps him close to the ground while he is deep in a Claude session or running back to back executive meetings. He can see a support conversation with a specific customer that needs product input, flag it, tag the right person, and turn it into a process improvement opportunity all in the same thread.

This is the thing most leaders lose as they get more senior. Proximity to the real problems. The scanner gives it back without adding hours to the day.

Chiefys

The third tool is the most strategic one. Chiefys is a bot that Customer.io‘s CEO Colin built. It lives in Slack and has one job. It holds the corpus of Customer.io‘s official company documents, the operating model, the strategy docs, the things the company actually runs on, and it checks new work against all of them.

When Matt produces something new, like a pricing philosophy, he can run it through Chiefys. It will tell him whether what he just wrote contradicts anything already established across the company’s core documents.

It works in both directions. New document contradicts an old one, Chiefys flags it. Old documents go stale because the company shipped something new, Chiefys flags those too. Every leader has experienced the pain of getting alignment on something in January and watching that document become invisible by April because three other things changed around it. Chiefys is the accountability check that nobody has time to do manually.

Get Transcript

Where to find Matthew Wensing

• 1:1 Video Consultation

• LinkedIn

• X

• Customer.io

Related content

Podcasts

1. PM’s Guide to Claude - YouTube | Spotify | Apple

2. How to Become a Builder PM - YouTube | Spotify | Apple

3. We Built an AI Product Manager in 58 mins - YouTube | Spotify | Apple

Newsletters

1. How to Build Product Strategy with Claude Code

2. My Claude Code PM OS

3. Team OS

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Refer a friend

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Bolt.new - Ship AI-powered products 10x faster

2. Product Faculty - Get $550 off their #1 AI PM Certification with code AAKASH550C7

3. Customer.io - Send smarter messages using your product data

4. Ariso - Ship AI agents and features faster, with fewer regressions

5. Jira Product Discovery - Plan with purpose, ship with confidence

Today’s episode

Six months ago, I showed you Codex is the best way to use ChatGPT for PM work.

Those who tried it have had great reviews:

But most of the content out there for PMs is all about Claude Code (myself included). So I wanted to get you the advanced setup for Codex, that’s up to date as of June 2026.

That’s what today’s episode is - an inside look at expert usage of Codex, from an OpenAI PM himself.

Abhi Muchhal is an International Growth PM at OpenAI and before that was at Meta and Nubank. He opened his actual setup on camera: the harness… everything. Then he covered how to crack a role like OpenAI yourself.

Apple Podcast

Spotify

• To get access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

• To get access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

Newsletter Deep Dive

Thank you for having me in your inbox. Here is the complete guide to using Codex like a senior PM:

1. The PM’s Ultimate Guide to Codex

   • The Codex harness setup

   • Automations that run while you sleep

   • Prototyping without a designer or engineer

2. How ChatGPT has Grown to 1B monthly active users

3. [BONUS] How to break into AI PM at a company like OpenAI

Subscribe now

1. The PM’s Ultimate Guide to Codex

In this section, we cover:

• How to setup Codex or your agent’s harness

• The 3 automations Abhi swears by

• How to build prototypes in Codex

The Codex harness setup

As I wrote in my Ultimate Guide to ChatGPT Codex, the harness is what separates a one off interaction from a persistent system. It is the connectors, the folder structure, the permissions model. Without it, you are just using a very expensive autocomplete.

Data connectors

The first thing to set up is every data source Codex needs to know about.

At OpenAI, the international growth team was pulling from seven or eight sources. Tableau dashboards. Databricks dashboards. Different tools, cadences, and formats.

Connect all of them.

The permissions model

Give Codex three levels of permission:

1. Reading tasks get full autonomy.

2. Synthesis and writing drafts get full autonomy

3. Anything going to another human needs your approval.

Skills, skills, skills

One growth engineering team at OpenAI got tired of running experiment reviews by hand. So they built a skill. Point it at a StatSig experiment, and it writes the hypothesis, monitors the experiment, and generates a postmortem and recommendation automatically.

That is what skills are. Reusable workflows, written once, triggered by name. PMs skip this layer entirely because building a skill feels like an engineering task. It is not.

When asked who builds these skills, Abhi said:

“The beautiful thing about Codex is that the person who cares the most is the one who makes the skill. It doesn’t matter if it’s an engineer, an analyst, or even a PM. I’ve made some skills as well.”

I’ve been hammering this home for months. Good skills are everything!

Abhi’s Top 3 Automations

Abhi swears by three automations:

Automation 1 - Daily Slack triage

Abhi works across time zones. By the time he wakes up, his Slack has 200 unread messages across a dozen channels. He was missing important things because there’s no priority filter for him.

So he built a Slack inbox triage that runs every day at a fixed time. It knows which channels matter, which senders are priority, and what kinds of messages need a response. It delivers detailed outline, things you have not read that you should, and things you have not replied to that are waiting.

The key prompt structure:

Automation 2 - A dashboard that updates itself

Abhi’s team was pulling from seven or eight different sources to understand how ChatGPT was growing across priority markets.

So he built a web app that pulls from all of them, refreshes every morning at 9:30 AM, and gives you the combined view plus the key takeaway. Country tabs. Top line metrics. Codex generated strengths and risks for each market relative to peer countries.

Let’s take a moment here and ask yourself - what are the sources you check manually every week, and what would it look like if they were in one place with a machine generated TLDR waiting for you?

Automation 3 - A weekly update

The weekly stakeholder update is the most time consuming, least value added task on most PM calendars. It pulls from Slack, Google Drive, Notion, and the same dashboards powering the growth view.

So he automated the synthesis. Codex pulls from every source, writes the first draft, and posts it to a channel he owns. He reviews it, edits the things that need his judgment, and sends it to his stakeholder group.

Prototyping without a designer or engineer

Before AI, getting a prototype in front of engineers meant writing a PRD, convincing a designer to prioritize your request, doing two or three rounds of mocks, and then finally handing something over. That whole loop could take three weeks for a simple feature. OpenAI cuts the loop, as Abhi described it:

Step 1 - Replace the PRD with a prototype plus companion doc

A prototype with a companion doc that addresses the obvious questions gets to the same place faster and creates a better conversation.

The companion doc answers ten questions. Why are we doing this. What must work for V1. What are the edge cases. What does success look like. It’s more like an FAQ that lives beside the prototype.

Showing a working prototype changes the quality of the conversation. Engineers stop asking what you want and start asking how to make it better. That shift leads to a huge leap in productivity.

Step 2 - Spec the output before you prompt

Vague prompts return vague prototypes. Before you send a single message, write down three things:

Paste that as your first message. Everything Codex builds follows from that. The more specific the spec, the less time you spend in correction loops.

Step 3 - Preview in app before handing off

Codex can now spin up a local preview inside the interface. You do not need to open a browser. You do not need to configure localhost.

Run the preview. Check the layout. Ask Codex to match a specific brand aesthetic if needed. Use Playwright to verify the fix. Then hand it to engineering team.

The biggest failure mode at this stage is showing an engineer a prototype that crashes in the first click. Test it before you show it. Codex will catch most of the surface level issues itself.

Getting to 80% without engineering time is non negotiable.

2. How ChatGPT has Grown to 1B monthly active users

There is an assumption baked into every AI PM playbook written in the last two years. The user has a desk. The user reads English. The user has a knowledge worker job that involves staring at a screen.

Abhi’s job is to break that assumption.

As International Growth PM at OpenAI, he is responsible for markets where knowledge workers are a single-digit percentage of the working population. India is under 10%. Brazil is 10-20%. These are OpenAI’s fastest-growing markets.

He emphasized three strategies the team implemented that I want you to notice:

Building for language diversity

The people in Bangalore opening a bookstore are not thinking in English. They want a website in Kannada. They want invoice templates in Hindi. They want a YT shorts caption that lands for an audience in Tamil Nadu.

Codex can generate, translate, and format across languages at a level that was not possible 18 months ago. Character rendering in Indian scripts, Japanese manga-style layouts, multilingual infographics.

Synthesizing information that lives on WhatsApp

In markets like India and Brazil, business happens on WhatsApp.

With checking everything manually you can now point Codex at your WhatsApp desktop app, ask it to summarize what you missed, identify action items, and draft a reply based on your calendar availability. In the episode, this entire flow took just over a minute. The reply appeared in the WhatsApp composer, pre-typed, waiting for you to hit send.

This is a workflow every PM building for international markets should understand.

Validating decisions in markets you are not in

The hardest part of international growth PM work is not building the product. It is understanding whether a feature that works in the US will land in Southeast Asia or West Africa.

Codex cannot replace user research. But it can synthesize the data you already have and surface the gaps faster. In growth work, the teams running the fastest are not collecting more data. They are synthesizing what they have with more precision.

For regulated markets or markets where your data pipeline is limited, the synthesis layer matters even more. If your Databricks dashboards are behind a compliance firewall, a Codex skill that only reads from approved exports still beats manual review by hours.

The PM who understands where their product actually lives in the world is harder to replace than the PM who only knows where it is built.

3. How to break into AI PM at a company like OpenAI

The question I get most from PMs reading this newsletter is some version of - how do I get from where I am to OpenAI, Anthropic, or a frontier lab?

I wrote a complete guide on breaking into OpenAI here. Since then, I have talked to four OpenAI PMs on this podcast.

Three key ingredients come through from the 4:

Key Ingredient 1 - Deep PM fundamentals, not just AI familiarity

Every frontier lab PM I have talked to had a serious pre AI career. Not just a familiarity with the tools. Deep PM fundamentals. Structured thinking. Analytical decision making. Communication under ambiguity.

This should be reassuring. The PM skills you have built are not obsolete. They are the foundation. The AI fluency is what you build on top.

What I covered in my AI PM job search guide still holds. The comp at frontier labs is real, but the evaluation criteria are not that different from a strong senior PM role anywhere. You need to show you can own a problem end to end.

Key Ingredient 2 - You have to build something

Every OpenAI PM I have spoken to built something with the APIs before they were hired.

Not a side project that exists as a GitHub repo nobody has seen. Something that actually ran. Something you had to debug. Something where you discovered what Codex or GPT could not do and had to work around it.

The builder credential matters because it is the only way to know what the model actually fails at. You cannot interview your way into that knowledge. In Abhi’s case he built a Chrome extension for real time language translation, deployed it on the OpenAI API, and was demoing it at the time of their application.

One honest caveat here. This path is more accessible to PMs who have slack time to build and who can access paid API credits. If you are in a market where that is not trivially affordable, start smaller. A Codex CLI project on your own machine costs almost nothing to run. The goal is to understand failure modes, not to ship a production app.

Key Ingredient 3 - Speak the language of evals

This is the one that surprises most PMs.

Evals are how frontier labs measure progress. A rubric that defines what good looks like for a specific capability, a baseline measurement, and a goal to beat.

You do not need to have run 50 evals to talk about this fluently. You need to understand why they exist, what they replace, and what a good eval measures versus a bad one. I covered this in depth in my LLM Judge guide and in the Ankur Goyal episode where we built one live on camera.

The PMs who get the farthest in frontier lab interviews are the ones who can say: here is a capability I care about, here is how I would measure it, here is how I would know if the model improved.

Every PM who can think in evals is already speaking the language of the companies building the future. That’s it for today, try to reread and understand the full workflow. See you in the next episode.

Get Transcript

Bonus. How to Get the Most Out of Codex:

Share

Where to find Abhi Muchhal

• LinkedIn

• X

Related content

Podcasts:

1. The Ultimate Guide to ChatGPT Codex

2. How PMs Ship 100K Lines of Code at OpenAI

3. Evals are the new PRD

Newsletters:

1. My PM OS

2. AI Agents Guide for PMs

3. How to Land a $300K+ AI PM Job

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Share

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Product Faculty - Get $550 off their AI PM Cerficiate with code AAKASH550C7

2. Bolt - Ship AI-powered products 10x faster

3. Customer.io - Send smarter messages using your product data

4. Ariso - Ship AI agents and features faster, with fewer regressions

5. Pendo - The #1 software experience management platform

Today’s episode

Most companies are still debating whether PMs should ship code. OpenAI is debating the best ways for PMs to ship code. They’re living in the future.

The builder behind a lot of that harness engineering is Ryan Lopopolo. He wrote the OpenAI post on harness engineering and runs a frontier team where PMs, designers, and engineers all ship using the same system.

The wild part for me? His PMs shipped around 100K lines of production code.

Did they open the IDE? Hell no! Their coding happened through PRDs, tests, docs, and harness rules. The model did the typing.

As someone who spent a decade in PM growth roles, I’ve seen how long it takes to move a feature from PRD in a doc to code in prod. For most companies, that latency is weeks, months, or quarters.

In Ryan’s world, it can be days, hours, even minutes. And the PM is inside the loop - not watching from Jira. So I had to understand how that works - and that’s today’s episode:

Apple Podcast

Spotify

• If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

• If you want access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

Today’s post

Here’s the guide to shipping code like PMs at OpenAI:

1. How the harness works

2. Ryan’s Frontier team case study

3. The new job for PMs, designers, and engineers

Subscribe now

1. How the harness works

You don’t need to copy every detail of OpenAI’s setup.

You do need the big pieces, because that’s the system you’ll be dropped into as an AI PM. Ryan thinks of the harness as the environment inside the repo that teaches the agent how your team builds software.

agents.md and the docs tree

Every repo has an agents.md file at the root. When they run Codex on that repo, this file is always in the model’s context. agents.md does two things. It explains the operating loop like read docs, plan, implement, run tests, and ask for review. Then it points the agent at the docs tree like markdown files with performance patterns, networking practices, user journeys, design guidelines, and past decisions.

Each run’s execution plan gets written back as implementation history. Design docs move out of slides and Slack into the repo. And tests fail if docs fall out of sync, which forces the agent to update them while editing code. The model needs a repo that it can search to show how your team works. As a PM, that’s where your decisions belong.

Tests and lints that enlist taste

Ryan’s team encodes taste and non‑functional requirements directly into tests and lints. No please remember I beg of you paragraphs. A few examples:

1. The build fails if any user‑facing markdown or HTML string uses straight quotes instead of curly quotes, because their design team is finicky about typography.

2. Business logic has to belong in the right modules. Tests enforce boundaries so a ball of mud doesn’t form.

3. When docs need to move together, tests enforce that they stay in sync.

Because the model is trained to make tests pass, each failure message is an instruction. You’re not arguing taste in Slack. The suite is the style guide. In growth roles, I learned the hard way that if something matters, you have to measure it and enforce it. Their approach is the same idea, just pointed at code and docs instead of funnels.

Review agents as virtual staff engineers

On the review side, they run a matrixed CI job that spins up specialized review agents, each with a persona doc:

1. frontendarchitect.md

2. reliabilityengineer.md

3. appsecengineer.md

That feedback goes back into the loop. The implementation agent uses it to fix the current change. The team updates the docs or tests so the same class of mistake doesn’t come back.

Observability and UI control

Finally, the harness gives the model eyes on the product. The agent can spin up a local observability stack (metrics, logs) and use it to debug like a human. With GPT‑5.5’s computer use, it can click through the app, inspect elements, and verify flows end‑to‑end.

The rule is simple; the agent has to prove the feature works through the same paths your users take. As a PM, that’s where your acceptance criteria also need to land.

2. Ryan’s Frontier Team Case Study

In mid‑2025, his team ran an experiment. Could Codex do the entire software engineering job for a new internal agent product?

The constraints?

1. Start from an empty repository.

2. Build an app that supports non‑engineering knowledge work inside OpenAI.

3. Let the agent handle on‑call triage, internal tools, and text‑heavy workflows.

4. No human is allowed to type production code. Engineers can only touch the harness.

By the end, the app had around 1M lines of code. It also had roughly 250K lines of markdown prompts inside the repo. And it ran as an Electron app with multiple agents coordinating summarization, repo gardening, skills distillation, and execution.

Any time the agent failed, the team asked what was missing in the harness that led to this failure. The codebase improved because humans improved the environment. Nobody jumped in and rewrote everything.

3. The new job for PMs, designers, and engineers

For most of my 10+ years in product, PMs were kept away from the repo. You were the human API between customers and engineering. You wrote PRDs, argued in sprint planning, and waited. The typing was someone else’s job.

Ryan describes a different world.

In his team, code is no longer the scarce resource. With GPT‑5, the model can generate code at scale. The difficult part is validating it, safely deploying it, and making sure it solves the right user problem. That means the harness (the environment of docs, tests, lints, review rules, and observability) is where work gets done.

Once the harness is strong, engineers aren’t the only ones who can write code. PMs and designers easily plug into the same system.

And literally everyone gets a new job description.

The new role for PMs

In late 2025, Ryan’s team wanted to explore a ‘skills system‘ so the agent could learn and reuse user preferences for data analysis. In most companies, this would kick off a chain of product reviews and engineering estimates. Here is what they did instead:

1. The PM wrote a markdown PRD that defined the skills library

   • How the agent should interview users about the metrics

   • How should it store and reuse that knowledge

   • How this should appear in the product surface

2. The team reviewed the PRD once in their weekly meeting.

3. By the end of the week, the feature existed in the app. The tests the PM had vibed into existence were passing.

The PM and engineers did not need a week of back‑and‑forth. The harness carried the intent into code. This worked because business logic was embedded in modules with high‑fidelity fakes. Also, the tests from the product side exercised the behavior.

The important bit: the PM’s leverage came from writing artifacts that the harness understands. If you’re a PM reading this, imagine your next feature:

1. Could you express it as PRD + tests + evals the agent can run on its own?

2. If you couldn’t touch the IDE at all, what would you have to write differently?

That’s the muscle you need to build.

Designers generate painted doors with

Early on, if a designer tried to ship a feature, they had to schedule agent runs inside their internal app. Then, to demo it quickly, they had to push a backend‑style cron system into the front of the stack. Usually, the scheduler logic ended up tangled inside frontend JavaScript.

Ryan looked at it and knew they’d regret it later. So decided to revert the change. Their team came out of that with a simple norm:

1. Designers own full painted‑door flows at the front of the app: the UI, the interaction pattern, and how scheduling is exposed.

2. Behind the door, the backend can be a no‑op plus instrumentation. They measure clicks, intent, and drop‑off before committing engineering time.

The harness then decides, based on data, which doors deserve a backend. So the JD for designers is to build high‑taste, high‑signal surfaces that the harness sees as experiments. A designer now must give the agent and engineers the best possible signal about what’s worth building.

Coding engineers became harness engineers

Ryan’s definition of the modern engineering job is very close to how I think about great growth engineers. Your value is the leverage you create for everyone else.

In his team, the primary output is leverage. Every engineer is responsible for adding tests, docs, CI rules, and observability that let many concurrent agents work. Internally, they use a simple mental model. Think of it as you just hired a cluster of interns named Codex. Your job is to manage them.

Emotionally, that’s difficult. Many engineers (including Ryan) grew up measuring their contribution in commits and lines of code. Letting the model do the typing can feel like losing the craft. But he argues that the craft moved. The new skill now is who can build the harness that makes GPT‑5 a strong teammate for this particular product and company?

“Every engineer on the planet now has hundreds or thousands of concurrent hands on keyboards, modulo token budget.”

That’s the world your PM career is walking into. You’ll be working with engineers who measure themselves by the systems they build.

Why should YOU care? Does it matter?

Most teams use AI to speed up function‑level coding but keep the rest of their software development lifecycle unchanged. Ryan’s world is what it looks like when you redesign the system around agents that can do the full job. The differentiator is how much of your team’s judgment is embedded in your harness.

If you’re a founder or product leader, that’s what AI strategy now means. It’s how you design the engineering system your PMs and designers plug into, so they can ship.

If you’re a PM, this is the future job description. You’ll still need taste, discovery skills, and stakeholder management. You’ll also need to think like a harness engineer: what tests, docs, and rules would I write if the model were my team?

Get Transcript

[Bonus] Downloadable Takeaway

I turned Ryan’s six‑month harness plan into a one‑pager you can use as a checklist with your team.

Where to find Ryan Lapopolo

• X

• LinkedIn

• OpenAI

Related content

Podcasts:

1. How to Run Evals in Claude Code with Aparna Dhinakaran

2. How to Build a Full AI Dev Team in Claude Code with Gabor Mayer

3. This CPO Uses Claude Code to Run His Entire Work Life with Dave Killeen

Newsletters:

1. PM’s Guide to Claude with Pawel Huryn

2. How to Become a Builder PM with Mahesh Yadav

3. How to Build a Team OS in Claude Code with Hannah Stulberg

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

Check out the conversation on Apple, Spotify, and YouTube.

Brought to you by:

1. Superhuman - The fastest email experience ever. Get 1-month free with my link

2. Land PM Job - My 12-week AI PM + Job Search Course

3. Vanta - Automate your compliance. Close deals faster

4. Product Faculty - Get $550 off their AI PM Certification, code AAKASH550C7

5. Bolt.new - Ship AI-powered products 10x faster

Today’s episode

If you’re building in Claude Code, you should learn how to build evals in… Claude Code.

There are two major evals platforms in the market today. We’ve already had the CEO and founder of one on, Ankur Goyal of Braintrust.

Today we return with the other major platform, Arize. And we have on the CPO and founder, Aparna Dhinakaran.

Many of the smartest AI teams are running their evals on Arize - Uber, Booking.com, Pepsi... I even offer Arize in my bundle. So now we bring you the guide to using it.

This episode builds upon the basics I taught with Hamel Husain, Shreya Shankar, and Ankit Shukla. It even includes a segment that, if a candidate did it in an interview, Aparna said she would hire them on the spot:

Apple Podcast

Spotify

• If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, Relay.app, Magic Patterns, Speechify, Bolt.new and Mobbin - become an annual subscriber ($150), and grab Aakash’s bundle.

• If you want access to my AI PM customizations - PM OS, Job Search OS, and Prompt Library - become a founding subscriber ($250).

Newsletter Deep Dive

I’ve put together the complete guide for running evals in Claude Code.

1. Evals don’t have to take forever and be hard

2. Step-by-step, how to run Claude Code evals

3. How to build a self improving loop

4. The new PM operating system

Subscribe now

Save this. A 3-step playbook on how to run Claude Code evals with exact commands.

Share

1. Evals don’t have to take forever and be hard

In the past, I’ve taught you the ways of Hamel Husain and Shreya Shankar who said that you need to look at the traces yourself, categorize them yourself.

In practice, what this means is a lot of PMs skip evals because it takes too long.

That’s a mistake. You can actually use Claude Code to do all those steps.

Here is what changed.

The old assumption - Evals need human-led error analysis

The old model assumed you had to be the one reading every trace. You needed to understand what went wrong before you could write an eval that tests for it. That meant the eval quality was capped by how much time you had.

That assumption made sense when models were weaker. It does not hold the same way anymore.

The new reality - Claude can suggest your first eval

When I asked Aparna, she said:

“I think it’s okay to start with Claude suggesting what a good suggestion of an eval could be. These models have gotten so good. Having it go through and look at your answers and suggest, that probably is something you should flag and look at. I would trust it. I would trust it as a first pass.”

This is not about replacing rigorous eval work. Hamel and Shreya’s methods still produce the most calibrated evals over time. But you do not need to start there. Claude gives you a working v0 in minutes. That v0 gets you data. That data gets you something to actually improve.

Aparna demoed “vibe based evals” using Claude Code. Here’s how it works.

2. Step-by-step, how to run Claude Code evals

You have an agent running. It is producing output. You have no idea if that output is accurate or just confident looking.

Here is exactly how to change that in one session.

Step 1 - Install the Arize skills into Claude Code

Open Claude Code. Run one command:

Claude Code now has access to instrumentation, eval suggestion, and improvement skills, you do not need to write any eval code yourself. You just ask Claude Code in plain English what you want. It calls the right skill for the job automatically.

The instrumentation skill is the first one you need. It looks at your codebase, identifies every LLM call and tool call, and wires them to Arize for tracing.

Step 2 - Ask Claude to suggest an eval

Once you have traces, ask:

Claude looks across your traces and returns candidates. In Aparna’s live demo it surfaced three:

• Report groundedness. Do the issues cited in the final PM report actually exist in the source data?

• Priority alignment. Does the P0/P1 ranking in the report match the highest-scored issues?

• Report actionability. Is the output something a PM can act on the same day?

Good starting point. But these are output level evals. You want to go one layer deeper.

Step 3 - Get specific about what you actually care about

Ask something more targeted:

Claude runs this across your spans. In the demo it flagged four failure categories: bug priority scoring, feature request scoring, legacy scoring system conflicts, and low priority edge cases.

Now you have something real. Specific categories of failure, with specific spans attached. That is what you need to improve the agent systematically.

You cannot fix what you cannot name. The eval names it for you.

3. How to build a self improvement loop

Your eval is running. It is flagging failures. The natural next question is: now what?

Most PMs stop at the eval. They have the signal. They do not build the system that acts on it. That is the difference between a PM who runs evals and a PM who has a self-improving agent.

The loop - three moves, on a schedule

The loop is not complicated. It is three moves that run automatically on a cadence you set.

Move 1 - Fetch everything the eval flagged as wrong. Claude pulls every span from the last cycle where the priority accuracy eval fired.

Move 2 - Find the pattern. Claude groups those failures by category. Bugs being underweighted. Long feature requests getting inflated scores. Edge cases involving third party integrations always landing wrong. The categories tell you where the agent’s logic is broken.

Move 3 - Propose a fix. Claude suggests a specific prompt change or scoring adjustment that would address the most common failure category. A concrete edit you can review and approve.

To set this on a schedule, ask Claude:

Claude spins up the equivalent of a cron job. Every day, or every week, your choice, the loop runs. It fetches failures, groups them, proposes a fix, and waits for you.

The human stays in the middle, always

This is the part that gets skipped in the excitement of automation. The loop is an accelerator. It is not a replacement for your judgment.

Eval changes need your sign-off before they run. Agent changes need your sign-off before they ship. The loop proposes. You approve. That is the design.

Aparna put it cleanly:

“Get data in, get an eval set up, give it criticism and let it go run on a loop.”

The criticism is still yours. The speed is Claude’s.

What compounds and what does not

The teams that build this loop are getting smarter with every cycle. Each round of failures produces better calibrated evals. Better evals produce more accurate agents. More accurate agents produce fewer failures. The signal to noise ratio improves over time.

The teams that skip the loop keep starting from the same baseline every sprint.

4. The new PM operating system

Step back from the mechanics for a second.

What you just learned is not a new feature to add to your existing workflow. It is a different operating system for how you do the job.

What the old operating system looked like

Monday morning. Open Linear. Read 40 new issues. Mentally rank them. Write a summary for standup. Two hours gone before you have typed a single Slack message.

Thursday. Pull up Gong. Watch three customer calls. Try to find the pattern. Write a themes doc that gets skimmed in a meeting and never opened again.

That cycle has a ceiling. There are only so many issues you can read. Only so many calls you can watch. Only so many sprints in a quarter.

What the new operating system looks like

The new operating system runs differently. Your agent already ran overnight. It pulled the latest issues, scored them by severity, recency, and reaction count, and wrote the PM report. It is sitting in your repo when you open your laptop.

Your job is a five-minute scan. Do you agree with the priorities? When you disagree, that disagreement is your next eval.

As Aparna said:

“If you are doing things the same way you were doing them last year, you have not caught up yet.”

That’s it for today. If you have any doubts, Aparna demoed the full process in the episode. Run the simple commands shared above. Watch Claude Code vibe it.

See you in the next episode.

Get Transcript

Where to find Aparna Dhinakaran

• LinkedIn

• Arize

• X

Related content

Podcasts:

1. AI Evals with Hamel Husain and Shreya Shankar

2. Evals are the new PRD with Ankur Goyal

3. AI PM Crash Course with Aman Khan