The Feature Results Writeup: An Advanced Guide

How to Break-down Features Results Like a Vet

You put a ton of time into a feature. You ship it. And, the metrics don’t move. Worse, the feature actually starts messing with your broader metrics.

I can’t even count how many times this has happened to me.

It’s not as unusual as you might think.

Most Ideas Fail

More than 50% of ideas fail - at trillion-dollar companies like Amazon and Microsoft. Airbnb has an insane 92% failure rate.

It’s just a fact of product: if you’re rigorous, not everything you ship is going to work.

The key is to be able to catch the problematic results so you can learn from them.

Where is the Content?

It’s basically a meme at this point. I love to write deep-dives about the content no one else writes about in Product Management.

But there’s seriously no good content on how to write up the results of a product launch:

I was pretty surprised about this, given we have the entire field of product analytics, but it appears this content has just not been widely written about yet by the normally hungry PM content writers.

Enter today’s piece.

Today’s Roadmap

This is your ultimate guide to the feature results writeup. Over 4,300 words, we cover:

1. The Underestimated Power of a Writeup

2. The Anatomy of a Killer Feature Results Writeup

3. Most Common Mistakes

4. The Art of Data Storytelling in a Product Org

5. The Stages of a Writeup

6. The Feedback Loop

7. Your Toolkit for Success (Including a Template)

---

1. The Underestimated Power of Writeups

A feature results writeup is not just a ‘nice-to-have.’ A well-crafted writeup consolidates all the data points from the development process. It’s the crucial wood to create your fire for your next quarter, your roadmap.

• Did the feature have a positive impact? Great, let’s allocate more resources to optimize and scale it.

• Did it flop? No worries, the writeup will help us decide whether to refine or retire it.

And it doesn’t stop there.

A Tool for Accountability

The feature results writeup is the enforcement mechanism for accountability within your team. More than 50% of product launches fail, but most PMs or product leaders don’t even know it. They just don’t rigorously analyze it.

I recommend that you have a table in the document to include analytics and PM sign off on the results. It should be just like a feature released into the wild: you have to sign your name in ink on the results.

By making it routine to document and assess the results of each feature, you’re setting the expectation for clear, measurable objectives from the get-go. This fosters a results-oriented mindset from everyone - from product to engineering and design.

The Only Path to Organizational Improvement

When you’re planning out your next quarter, your results write-ups become the fuel for your fire. It’s tempting to think you, as the PM, will remember the results. And maybe that’s true. But the organization won’t.

Having a culture of good write-ups is the way to create an institutional memory of what happened. Two years later, when someone is looking to move the metric, there’s actually something to look back upon.

The writeup is not just a reflection of the past. It’s a stepping stone for the future.

Should You Have a Writeup Without an A/B Test?

If it matters for your metrics, yes. Of course, none of the things you’re seeing in the data are hard conclusions without an A/B test. But what I recommend you do is you find stable metrics that haven’t been moving that you intended to move.

In fact, I usually, for statistical hygeine, ask my PMs to only look at one’s they have identified beforehand. Of course, you still haven’t achieved independence, but at least looking at the data for something you predict allows you to have a bit more faith in the takeaways.

So What Shouldn’t You Have a Writeup For?

Anything that you don’t expect to move the metrics.

There are tons of things like bug fixes, support requests, addressing tech debt, and design upgrades you’ll do as part of normal product hygeine. These things don’t need the metrics-focused writeup we’re discussing here.

Wrapping Up Section 1

The feature results writeup is not just a post-mortem document. It’s your strategic powerhouse. It guides informed decision-making, fosters a culture of accountability and continuous improvement, and serves as a repository of insights for future projects.

So, the next time you ship a feature, take a moment to reflect, document, and appreciate the ripple effects of a well-crafted feature results writeup.

---

Did you know many readers expense this newsletter to their learning & development budget? Here is an email to use to get it approved.

---

2. The Anatomy of a Killer Feature Results Writeup

There are five key elements to a killer write-up:

1. An order that helps

2. The good, the bad, and the ugly

3. The 5 Why’s

4. The lessons learned

5. The next steps

Let’s break down each one.

Feature 1 - An order that helps

The ordering of your writeup is one of its most important elements. There’s three narrative structures people like to take: