Customer Interviews: Advanced Techniques

PM's Most Fundamental Skill

Find me a product manager who consistently isn’t making impact while given the resources, and I’ll find you a product manager who doesn’t know how to talk to their customers.

The gurus can make product management seem like it’s incredibly complex, but it’s actually really simple. You need to solve problems for your users. To do this, you need to:

1. Talk to them frequently

2. Know how to build product features that solve their problems

Yes, I’ve of course written a metric ton about product management rigor - from PRDs and roadmaps to impact sizing and product reviews. But ultimately, that’s all window dressing for good product thinking that PMs in medium to large businesses need to do.

If you’re a solo PM or a solo founder, most of that is just overkill. In the world of 80/20 Pareto optimization, none of that rises to your 80%.

Instead, what matters most is learning how to get the right information from your current, prospective, and churned customers.

If you don’t get this right, you end up one with one of two types of problems.

Problem Type 1: Just Doing What Users Say

The most common trap PMs fall into who regularly talk to customers is they process the signals from customers al wrong.

It’s like as if Tim Cook decided to build the Apple based on user petitions and suggestions:

If you are building a B2B product or aren’t your product’s ideal customer, it can be hard to see when a product looks like this.

Just think of someone who didn’t actually use an iPhone designing it. They hear users want a RCA port, so they add it…

That’s what PMs can be tempted to do when talking to customers if they don’t do it right.

Problem Type 2: Doing What You Think

Problem type 1 is a problem of listening to customers too much. Problem type 2 is a problem of not listening to customers enough.

PMs get so busy with PRDs, code, and Figma files that they forget to talk to their customers.

Sadly, this is more than 60% of product orgs I encounter these days.

The problem is culture. Usually, the Product org’s culture doesn’t make talking to customers easy for PMs - that’s how it was at Affirm, for instance.

But sometimes, even when it’s easy, the culture doesn’t reward it enough. Executives reward PMs who build great presentations and write good PRDs over PMs who talk to customers and build the right products.

As a result, everyone in the company gets caught up in a cognitive trap, “The Curse of Knowledge.” We think that our customers have the same background knowledge we do.

But they sure don’t. And the business’s performance declines as a result of mismanagement.

Today’s Piece

To avoid both these problems, product leaders and PMs need to talk to customers often, and talk to them well. In fact, it might be the most important skill in Product Management.

So, in today’s piece, we’ll walk through the entire lifecycle of customer interviews. Over 6,000+ words, we’ll cover my best practices in each stage:

• Part 1: Pre-Interview

• Part 2: Mastering the Interview

• Part 3: Advanced Analysis Techniques

• Part 4: Implementing Insights into Action

It’s the value of all the books I’ve read and the customer interviews I’ve done in 15+ years in one article.

---

Section 1: Strategic Preparation

In a world full of products that solve non-problems, the customer interview is your most potent weapon against waste: wasted time, wasted effort, and wasted resources.

However, as I've learned from countless interviews, all conversations are not created equal. The effectiveness of your customer interviews actually is rooted in how well you’ve prepared. You need to know:

• 1.1 When to interview

• 1.2 Who to interview

• 1.3 How to develop a great interview guide

Let’s go through each.

1.1 When to Interview

Customer interviews are not a one-and-done deal.

They should be conducted at multiple stages throughout the product development cycle.

Stage 1 - Before Development

This is the discovery phase. As a PM, you're not selling anything; you're learning. Your goal is to understand the customer's world, their problems, and the value (if any) your solution could provide.

Here, you'll ask about their life, their problems, and their past efforts to solve those problems. This is the stage for opportunity interviews - where you are not validating ideas but uncovering problems.

Stage 2 - During Development

This is the validation phase. You have a hypothesis about the solution to your customer's problems. Now, it's time to test it.

This is the stage for solution interviews, where you validate if the proposed solution indeed solves the customer's problem. Here, you'll ask for feedback on specific features, functionalities, and the overall user experience.

Stage 3 - After Release

This is the iteration phase. Your product is live, and you're collecting quantitative data on user behavior. However, quantitative data only tells you what is happening, not why.

Here, you'll ask for feedback on the user experience, uncover unexpected use cases, and discover potential areas for improvement or expansion. Post-release interviews are crucial to understand how your product fits into the customer's life, what can be improved, and what should be iterated.

When to Interview vs Use Different Research Techniques

The user research toolkit is vast. Don’t just interview for everything.

Instead, choose the best tool for each job:

1. Surveys: Use when you need quantitative data from a large audience to identify trends or validate hypotheses.

2. Observations: Use when you need to understand natural user interactions with your product in real-world scenarios.

3. Usability Testing: Use when you need to evaluate the user-friendliness of your product by observing users as they complete specific tasks.

4. Focus Groups: Use when you need to understand group dynamics, conflicting opinions, or gather a range of perspectives on a specific topic.

5. A/B Testing: Use when you need to compare the performance of two different versions of a design or content element to make data-driven decisions.

6. Heatmaps: Use when you need to visualize how users navigate your website or app, such as where they click, tap, or hover.

7. Interviews: When the above doesn’t apply, use when you need deep insights into user needs, motivations, and challenges, or to explore topics that surfaced from other research methods.

1.2 Who to Interview

Your mom, your friends, and your colleagues are probably not your target customers, so don't ask them for feedback (or at least, take their feedback with a grain of salt).

Instead, focus on these groups:

• Current Users: Those who use your product and can provide insights on its strengths, weaknesses, and areas for improvement.

• Potential Users: Your target audience who are not using your product yet. They can provide insights on their current challenges, needs, and expectations.

• Churned Users: Those who used your product in the past but have stopped. Understanding why they left can provide critical insights into potential barriers and pain points.

Selecting the right interviewees is not just important; it’s everything.

1.3 How to Develop a Great Interview Guide

An interview guide is not a script; it's a roadmap. It's a list of topics and questions that you want to cover during the interview, but it's not set in stone. The conversation should flow naturally, and it's okay to deviate from the guide if the conversation takes an interesting turn.

Here's how to develop an effective interview guide. The basic process is just three steps:

• Step 1 - Define the Objectives

• Step 2 - Identify Key Topics

• Step 3 - Organize Questions Logically

But of course it’s not as simple as that. The devil is in the details. I would call out four key philosophical concepts to guide your guide.

Guiding Philosophy 1 - Avoid Leading Questions

Leading questions can steer the interviewee in a particular direction and may result in biased answers.

For example:

"Don't you think our new app feature makes things so much easier?"

This question is leading because it implies that the new app feature does, in fact, make things easier, and it prompts the respondent to agree.

You don’t want to use these. Instead, you want to ask neutral questions that allow the interviewee to share their genuine thoughts and feelings.

A better way to ask the question would be in a more neutral and open-ended way, such as: Neutral Question:

"How has your experience been with the new app feature?"

This question doesn't imply any particular answer and allows the respondent to express their thoughts freely, whether positive or negative.

This is the power of actually doing the pre-work to have an interview guide.

As I mentioned in the intro, even experienced practitioners often make these mistakes - leading questions is top amongst these. Experienced practitioners become more busy and thus decide to just jump into the interview and use their extensive experience.

But they don’t have the time to look at the questions they’ve written down and remove the leading questions. So they end up blurting them out.

Guiding Philosophy 2 - Make sure all the questions pass “The Mom Test”

Of the ~20 books I’ve read on customer interviewing, “The Mom Test” is the name I always remember.

So we’re bringing it to our Product Growth Interview Guide Guiding Philosophies.

What is the Mom Test?

Well, your mom loves you. She wants to make you feel great. She wants you to succeed.

You need to ask questions that even your mom wouldn’t lie to you about. Because customers in interviews are liable to be just like your mom - lying to your face (unconsciously) to make you like them and be happy.

Few interviewees want to make your day worse. That’s why they’ve agreed to the call.

As Rob says in the book, “everyone is always lying to you.” So make your questions one’s that are about historical facts, not future hypotheticals.

Guiding Philosophy 3 - Have a Visual Component

People, in general, are too polite to tell you your idea is bad. Prototypes, production, or beta versions of your app are one of the best antidotes. They are your opportunity to spot the 'polite objections' and get to the real issues.

Interactive prototyping tools are a game-changer. They accomplish three things particularly well

1. Visualize Concepts: Show the interviewee a visual representation of a new feature or concept you are considering. This can be a sketch, wireframe, or interactive prototype.

2. Gather Feedback: Ask the interviewee for their feedback on the visual representation. What do they like? What do they dislike? What would they change?

3. Facilitate Discussion: Use the visual representation as a starting point for a deeper discussion. For example, if the interviewee suggests a change, ask them to elaborate on why they think that change is necessary.

If you can, incorporate some type of prototype. But even if you can’t use an interactive prototype, show them a napkin sketch, or have them use the existing product.

You rarely want to have an interview which is just asking someone a bunch of questions.

Guiding Philosophy 4 - Leave Time for Probing Questions

While you have a guide, you must incorporate probing questions.

Probing questions encourage the interviewee to elaborate on their answers and provide more detail. For example, if the interviewee mentions a problem, ask,

“Can you tell me more about that?”

or

“Can you give me an example?”

These often don’t appear on the interview guide, but are where the insights actually from.

So the takeaway for your guide is to leave enough time to incorporate lots of probing questions.

So many people who don’t make a guide have tons of questions they wanted to ask that they don’t get to. So the win from a guide is being able to narrow down what you can actually accomplish.

You actually leave time for the conversation and prioritize accordingly.

Putting together everything we’ve discussed in section 1

I hope I’ve convinced you: customer interviews are a cornerstone of successful product development.

But, their success lies in strategic preparation, selecting the right interviewees, developing a thoughtful interview guide, and using interactive prototyping tools to facilitate deeper discussions.

It’s this strategic approach - not lackadaisical interviewing - that will ensure you gain the critical insights necessary to develop a product that truly meets your customers' needs.

---

Section 2: Mastering the Interview Process

Now let’s break down what to do when you’re actually in the interview. We’ll cover:

• 2.1 Opening the Interview for Success

• 2.2 How to Actively Listen & Adapt

• 2.3 Key Mistakes Teams Make

• 2.4 Learning Iteratively