How to Use ChatGPT as a PM or Product Leader in 2024

The latest meta for using ChatGPT as the service has evolved and grown since its release nearly a year ago

AI won’t replace PMs. PMs who use AI will replace PMs who don’t.

This post might save you more than any other I’ve sent to your inbox.

It’s hard to overstate how important of an innovation ChatGPT is. Released just 1.5 years ago - November 2022 - it can feel like it’s been around forever.

But, really, social media got over-inundated with posts about it last year. We all ended up with a bit of nausea just seeing, “10 prompts for ChatGPT.”

The cringe social media posts didn’t mean the product got worse, though.

ChatGPT Keeps Getting Better

If anything - the Human Reinforcement Learning is driving ChatGPT to get better and better.

I recently met with a VP of Product who said that he was “stunned” at how much better GPT-4 is than it is now compared to 10 months ago at its release date.

Not only does ChatGPT keep getting better, it’s going to keep getting better:

1. Up next is GPT-5, which Sam Altman described as his top priority.

2. AGI doesn’t seem likely in 2024.

3. Then, ASI (Artificial Super-intelligence) is within the horizon by 2031:

This makes ChatGPT a key PM skill

All these upcoming developments mean learning how to use ChatGPT well is like learning how to use Excel or SQL before everyone had to.

It’s a huge career advantage.

And unlike things like Excel or SQL, which have a huge learning curve, it has a medium learning curve - but very high usefulness for PMs:

The reason ChatGPT gets a 5 rating above SQL and spreadsheets? It can help you do wizardry in your spreadsheets and SQL.

---

But the meta keeps changing

In video games like Fortnite, players talk about the “meta.” It’s the ideal way to use functionality that’s widely available to everyone in the game.

The meta for ChatGPT keeps changing. OpenAI has an incredible velocity of shipping.

So if you’ve been using it the same way as 10 months ago, you may have missed some significant changes.

---

Introducing Carl

Carl is a Senior PM at GoodRX. All the way back in May of last year, I collaborated with Carl on an Advanced Guide to Impact Sizing. You all loved the piece!

So, we’re back with another post. We concocted the idea a few weeks back, and having been investigating the meta ever since.

You can find Carl on X, LinkedIn, or in his newsletter.

---

Today’s Post

Words: 8,671 | Est. Reading Time: 40 minutes

• The New Meta

• The Top 10 Use Cases

• The Top 7 Mistakes People Make

  Including…

  • What not to use ChatGPT for

  • How to manage hallucinations

  • How to avoid leaking company secrets

• The Extended Long-Tail of Use Cases and Prompts

• Alternate and Free Ways to Access ChatGPT

• Key Use Cases for Product Leaders

• Tips for Debugging

• The Future

---

The New Meta

There’s almost been innumerable changes in the last 10 months. On top of the neural networks responding to the user feedback, the team at OpenAI ships incredibly fast.

The key changes - for users of ChatGPT Plus - that have happened with the product experience in the last 10 months are:

1. Search and code interpreter have been built-in to to the core experience

2. It is much better at understanding long context and mega-prompts

3. The model is much less lazy and will do things you ask

4. You can upload PDFs directly into ChatGPT

5. It’s easy to talk with it via Voice

What it means for you

This has lots of implications for how you can use it. We’ll cover them in glorious detail throughout this piece.

But a key one to highlight is:

• Short prompts are OUT

• Long prompts are IN

There has been a bit of a Yo-Yo with ChatGPT. As the GPT-3 meta developed, mega-prompts became clearly meta.

But, as GPT-4 was released, the model was not reliable for long prompts. This led to the proliferation of short prompts.

That advice is old now. And mega-prompts are back.

---

The Top 10 Use Cases

There’s unlimited ways PMs can use ChatGPT - and we’ll cover more at the end.

But these are the top 10 that should make you more productive today:

1. Draft a PRD

2. Write a SQL Query

3. Apply a Framework

4. Do a Design Critique

5. Create Your Prototype

6. Draft an Impact Sizing

7. Respond to Email Faster

8. Explain a Technical Topic

9. Create a Discussion Guide

10. Edit a Feature Results Writeup

Let’s go through each in depth.

---

Use Case 1 - Drafting a PRD

Under the old model of prompting, you might just ask ChatGPT to write a PRD.

But now, it really helps to get things ready for it.

The Prompt

This mega-prompt below is your template:

# Instructions

## Act as a an expert writer of Product Requirements Documents. Make targeted assumptions to draft the type of PRD that a designer or engineer picking up would have absolutely no questions about.

# Skills and Areas of Knowledge

## Possess a deep understanding of the product, and creating a product requirements document to be ready for engineering and design.

# Next Steps

## Draft the PRD and share the questions you have to make it even better. Make best guesses where I don’t specify based on what I do specify.

# Personality and style

## Act as someone who has a good strategy, UX sense, business sense, and is confident this feature will work.

# PRD Template

## Problem 

Describe the problem we are trying to solve in 1-2 sentences. Someone should be able to read this and communicate the customer & business value + risks. Highlight evidence or insights we you have to support this.

## High Level Approach

Describe the rough shape of how we might tackle the problem. This should be a few sentences max.

## Narrative

Share (hypothetical) stories to paint a picture of what life looks like for customers today. Describe common and edgy use cases to consider when designing the solution. 

## Goals

Describe high-level goals, ideally in priority order and not too many. Include measurable (metrics) and immeasurable (feelings) goals. Keep it short and sweet

### Metrics

Highlight specific, operationalized north star metric, secondary metrics, and guardrail metrics. Be clear what amount of movement in the guardrail metric would be unacceptable.

### Impact Sizing Model

Create a model taking the known information to calculate impact size against the key identified metrics, as well as relevant bottom-line metrics like revenue/profit. State the estimated impact on the north star and bottom-line metrics. Show the calculation steps to get there.

## Non-goals

List explicit areas we do not plan to address. Explain why they are not goals. These are as important and clarifying as the goals

## Solution Alignment 

Draw the perimeter. Do not force others to identify scope.

## Key Features

### Plan of record

List the features that shape the solution, Ideally in priority order. Think of this like drawing the perimeter of the solution space. Draw the boundaries so the team can focus on how to fill it in. Challenge the size to see if a smaller component can be shipped independently.

### Future considerations

Optionally list features you are saving for later. These might inform how you build now

## Key Flows

Show what the end-to-end experience will be for customers. Get detailed here screen by screen, not high level. Highlight what screens a user is on, what they click, and the result.

## Key Logic

List rules to guide design and development. Address common scenarios and edge cases. Make it so an engineer feels everything is thought through.

## Launch Plan

Define the various phases that will get this product to market, the purpose of each phase, and the criteria you must meet to move on to the next one. Put this into a table.

Highlight whether it will be an A/B test and how long would be needed for stat sig results based on north star metric goal.

Highlight risks and dependencies that can throw a wrench in timelines or progress (and ideally contingency plans). Create a table.

## Key Milestones

Highlight the key developmental and design steps to go. Put this in a table.

# PRD Known Information

<Add what you already have>

This template, adapted from our deep dive on PRDs and inspired by Kevin Yien, will get you a great output.

It works better if you can also add some chicken scratch to each section and iterate with Chat GPT.

Specific Example

You can see the actual chat conversation here.

In this example, we’re working at Gmail, and we’re working on a Superhuman-like messaging framework that only shows them e-mails to respond to soon.

At the end of the PRD known information section, I’ve added details to the prompt of what we know so far.

People prefer the Superhuman inbox, which is more selective, to Gmail's Primary inbox, which is more permissive and user-rules based. We know this because Gmail saw churn of 0.75% of users last quarter to Superhuman, the largest churn to competitor.

We want to create a Superhuman style inbox and notification system that helps people only see what they need to respond to now, now.

As a user, when I open Gmail into Primary I see way too many messages. Superhuman just shows me the one's I want to reply to.

We want to reduce churn, increase people's satisfaction and NPS of Gmail, and ultimately increase MAU of Gmail beyond the 1.24B it is now

We don't plan to redo the primary inbox. We need to figure this out in addition / enhancement / or toggle

Several VPs are obsessed with the idea, but we have to make it work.

The result is a pretty good PRD first draft. It takes your chicken scratch and gives you a first version to iterate upon.

In particular, what’s valuable is it does quite a bit of work to get the barebones of a document going.

Of course, the mistake would be to share this with the team.

When looking at it, we immediately as PMs will several improvement areas.

You have two options at this point: iterate on those with ChatGPT in conversation, or just replace them yourself.

To give ChatGPT feedback, a numbered list works best. For example:

Great stuff. Some feedback.

1. Primary goal should be reduced churn, secondary should be user satisfaction and NPS

2. And that should also flow through to metrics

3. We should specify what engagement with the feature metrics we are considering

4. It's unclear how this interacts with primary inbox and we should make that clear in key features and flows

5. It's unclear how users opt in and out and we should make that clear

6. It's unclear how the AI model is different so we should make that clear

7. Just assuming a 2% increase in MAU isn't as persuasive as giving a reeason and calculating it off the 1.24b we have.

This leads to a nice improvement over the prior PRD:

Both patterns work. But probably the ideal PRD workflow has become:

1. Chicken scratch

2. ChatGPT Draft to start

3. Edit it (or with ChatGPT)

4. Keep editing it as the PRD progresses

Once you see what ChatGPT has done, you’re not working from the blank page of your PRD template. You have something to react to and improve.

PRDs are one of the primary work products of PMs. Making them awesome is more important than doing them. ChatGPT can help the time to awesome.

---

Use Case 2 - Write a SQL Query

In the ideal scenario, we’d have access to all the data we need - and our friends in the analytics department can help with the rest.

But in the reality of the PM day-to-day, there’s lots of data questions that come up, that you would benefit from being able to query yourself.

A strong mastery over a tool like Snowflake can be a PM superpower. It certainly was for me at Epic Games, where I was able to bring data to every conversation and knew how to write SQL queries that would yield results in 1 minute - not 1 hour.

But that took me 1000s of hours over years to achieve. Now, PMs can do that just with ChatGPT.

The Prompt

# Instructions

Develop a SQL query.

# Skills and Areas of Knowledge

Proficiency in SQL, with a specific focus on fast-executing queries that look up a reasonable sample, use sub-queries when appropriate, use readable aliases, and have great notes.

# Data and Tables Overview

<Insert>

# Query Crafting

The query should: <Insert>

# Result Expectation

A comprehensive SQL query that provides a clear use of the data tables to result in the query craft, along with a brief explanation of each part of the query for better understanding and future tweaking.

You basically want to a result that has:

1. Great notes

2. Great explanation

3. And doesn’t get anything wrong

So this mega-prompt gives you an easy copy-paste.

Note: You do want to include information about your tables, in addition to your query goals, to get the best response.

If you aren’t familiar enough with the data tables, you can also put:

What types of data tables should I be looking for? Walk me through iteratively to get to the right tables.

Example

You can see the actual chat conversation here.

Let’s say you’re a growth PM at Facebook. Your team just rolled out an A/B test on a revamped notification system aimed at increasing DAU.

You want to check-in on the results.

Here’s the type of information you’d want to add to the query.

# Data and Tables Overview

Access to 'user_activity' table with columns like 'user_id', 'session_id', 'session_length', 'activity_date', and 'notification_version' (1176a or 1177b). The 'user_segmentation' table includes 'user_id' and 'segment_type'.

# Query Crafting

The query should:

• Segment users based on the notification version they received.

• Sample because we have absolutely massive data with billions of users.

• Calculate the average session length and daily active usage for each segment.

• Compare these metrics between the two segments to determine the impact of the new notification system.

The result is a nice query that uses CTEs and has all the explanation you could need to use them yourself.

What’s especially helpful is the guidance at the end on sampling and dates:

That’s what ChatGPT is the king of SQL advice. I spoke with a Senior PM who says he uses ChatGPT 2-3x per week for SQL.

That sounds about right.

You can also continue with ChatGPT to iterate on what further questions you have, or error messages you run into.

The numbered pattern we mentioned earlier works well for this:

Great. A couple things:

1. 1% sampling might be too big still

2. How can we add some other cool metrics to this

3. EG how do we cut the data by people who receive 1,2,or 3 notifications; and males vs females; young vs old

The result is a query that will run fast and segments the data a bunch of ways.

Those types of case when statements are a bear to write on your own. Even I usually end up running into an error or two while writing them, after decades of SQL.

But with ChatGPT, they’re easy.

And the best part: From there, you can just keep iterating for what you want.

Using ChatGPT for SQL dramatically increases the volume of insights you can have, because it decreases the time to insight.

---

Use Case 3 - Create Your Prototype

One of the most “wow” things to do is go from napkin sketch to working prototype for user testing in 2 minutes.

And ChatGPT can do that for you with this mega-prompt: