An Advanced Guide to Impact Sizing

One of the hardest parts of PM

In Today’s Newsletter:

• Tech Corner: Code Interpreter is Must-Have

• Career Corner: How to Include Introverts

• Paid Corner: Advanced Guide to Impact Sizing

---

ChatGPT Code Interpreter is a Must-Have Tool for Knowledge Workers

I wrote about code interpreter this week, sharing 15 use cases for how it will disrupt data science. The key takeaway was: everyone will become a data analyst.

I think it’s a tool that just about every knowledge worker should sign up for.

All you need to do is get ChatGPT Plus and get on the waitlist for Plugins. The earlier, the better - as it’s a long list. Many folks who signed up weeks ago have not gotten off of the list. But the wait is worth it.

In any type of knowledge work in 2023, you can do your job better if you are data-driven. “Data ends arguments,” as the old Facebook posters used to say.

Not only that, but data is one of the best ways to build credibility in your decision-making in 2023. People tend to trust people who have analyzed all of the data. If you can quantify what you’re talking about, it feels much more credible.

The code interpreter can take your data file and clean it, give you descriptive statistics, and perform analyses in seconds - all through a chat interface.

It’s a superpower for knowledge workers.

---

BTW - In case you missed it from the last few days of me on social:

• Careers: How to answer common interview questions & use ChatGPT to convert more interviews to offers

• Product Management: Questions to ask at the end of an interview (generic version)

• AI: The Latest in AI (2, 3)

Thanks & WELCOME to the more than 3,500 new subscribers this week 🤯

---

Be Inclusive of Introverts

Source: Liz and Mollie

Introverts are often the odd ducks left out at work. How can you build an inclusive environment for introverts at work? Here’s what I recommend.

Remember not everyone talks to think. Some people need time to think before they talk. Also, not everyone wants to share ideas by talking. Some like to use Slack or email.

Try this in your meetings:

❌ Ask, "What do you think about our H2 plan? Tell us now."

✅ Say, "We want to know your ideas on our H2 plan. Take time this week to think, then email me by Friday with your thoughts."

This helps people with different ways of talking and sharing. Try this too:

❌ Say, "Let's brainstorm ideas for our new project. Share your thoughts out loud."

✅ Say, "We're going to think of ideas for our new project. You can write them down, then we'll collect them and talk about them together."

This way, everyone can share their ideas in a way that feels good for them.

---

I hope you enjoyed those two bite-size pieces. Now onto this issue’s deep dive.

---

An Advanced Guide to Impact Sizing

It’s mind-blowing that people still estimate the impact of a feature using something like high/medium/low (for an established product). I used to do it, too, 15 years ago. 

Now the Baseline

But, nowadays, the gold standard is actually estimating the impact of a feature. Most high-performing product teams estimate the impact of a feature to their OKRs (eg, engagement metrics). 

And the advanced one’s also estimate the impact of a feature to their output metrics like revenue and profit. This helps you truly prioritize the features that are going to change the trajectory of your business.

Aside from the obvious benefits of helping your career due to higher impact, this also sets you up well to impress in product reviews. You can create an amazing presentation for your roadmap that looks something like this.

The trouble is - impact sizing is super hard. As one public company senior PM said:

It’s always rough. I’ve rarely seen someone do it in a really scientific way.

How can you do it better?

So, the question is: how can you size better? 

It’s one thing to do the hand-wavy PM thing and say, “I’ll estimate a 10% adoption of this feature.” But that’s only going to get you so far. Your impact will be much less predictable. And rigorous product leaders, engineering and analytics counterparts will question your decision-making.

It’s much better to have data-driven reasoning behind your product sizing:

• Based on the actual number of users estimated to see the feature

• With a high confidence estimate of adoption and engagement impact

• And appropriate assumptions to understand the top & bottom-line impact

So, you go to Google and GPT-4 for help.

There’s No Content

Google and GPT-4 fail on this type of content. 

If you Google it, there’s not a single article that actually walks through how to impact size different types of features and metrics. It’s quite a sad state of search results for “how to impact size product feature.”

If you BingGPT it, the results aren’t any better. It’s just a high-level explanation, with no tactical guidance. If you push BingGPT to impact size a feature, it will just give up altogether:

This type of impact sizing is something PMs just started doing in the last 10 years. As a result, there’s a dearth of good knowledge or best practices.

Enter Today’s Piece

Uncovering this content gap, for today’s piece, I’ve teamed up with one of my favorite Product Creators, and current Senior PM at GoodRx, Carl Vellotti. Together, we bring you: an advanced guide to impact sizing. 

We’ll cover:

• Templates to estimate the most common metrics

• How to set up experiments to identify and de-risk your riskiest assumptions

• Tips & tricks to get to the key data points

This is a ‘201’ level course for impact sizing. It’s a great fit for PMs new to the practice or product leaders looking to adapt it. It’s the type of post you’ll want to send to your colleagues.

Let’s get into it.

---