Idea validation vs. customer research (and how to do the better one)

Companies often assume what their customers think, feel, and want.

But that’s like trying to assess a home’s curb appeal from inside the living room or trying to draw a street-level view from the penthouse. You may get in the ballpark (if you’re really lucky), but you don’t have an accurate vantage point. Your perspective isn’t enough to tell you their perspective.

Our best clients get this and go directly to their customers to find out what’s going on. Then they use rich customer understanding to do things like find product-market fit in less than 18 months or increase daily active users by 62%.

And they’re not alone.

More and more, successful founders are ditching tactics like traditional idea validation in favor of more effective approaches like in-depth customer research.

Customer research vs. idea validation

Idea validation is proving your target market will pay for your idea before you build a product or business.

Customer research is learning more about customers’ wants, needs, fears, motivations, and more so you can build a product or service they love.

In the past, founders have used either method (or a mixture of both — many validation processes involve customer research) to build successful businesses. But in today’s SaaS world, validation is a much riskier approach.

Why?

Several reasons:

• SaaS is increasingly saturated. There is something like over 557,000 tech businesses in the US alone. Consumers have more options at more price points than ever before. In most cases, the issue isn’t coming up with something interesting, it’s coming up with something customers need and love. To do that, you have to deeply understand your customer.
• Validation starts on super shaky ground — your idea. In early business stages, your idea isn’t a fact. It’s a guess, and it changes rapidly. Starting here is like building a house on sand.
• Validation sets out to prove where customer research sets out to learn. The way most founders run validation looks a bit like this: typing a specific query into Google then pouring through tens of articles until the find the one that proves their point. In other words, validation is particularly prone to confirmation bias, where you search for and interpret data as support for your belief.

In software today, companies with the best customer understanding (not the best idea) are the ones who win. We’re in the age of the consumer and, to put it bluntly, consumers care very little about your product and a whole lot about the value you’re delivering. The companies who understand what customers value and why are the ones on the top of the charts (think Apple vs Sony, Amazon vs Borders, Netflix vs Blockbuster)                             

Tl;dr idea validation gets things backward. It starts with an idea, where the best starting place is your customer.

Why profitable businesses start with customer research

A while back, we spoke with Amy Hoy (founder of Noko and co-founder of 30x500) about growing businesses in a recession, she outlined the approach she’s been teaching since 2010 — in good and bad times.

It’s an approach that has helped founders cross $40k in monthly revenue, 10x their business, and cross major revenue milestones.

That process is:

1. Choose an audience that pays for things
2. Study them to find out what they do
3. Build something you know they will buy (based on their actual buying behavior)
4. Build trust with educational content
5. Don't quit your day job until you've got a stable income stream

Did you catch those first two steps of selecting and studying an audience?

That’s customer research!

And it’s not just folks who follow Amy who see results from focusing on the customer:

• Deloitte and Touche found customer-centric companies are 60% more profitable.
• Coschedule found marketers who do audience research at least once per year are 303% more likely to hit marketing goals, while successful marketers are 242% more likely to conduct audience research at least once per quarter.
• McKinsey cites brands that improve the customer journey see revenue increases as much as 10-15% while lowering service costs by 15-20%

Why?

Because customer research helps answer questions like:

• What personal or circumstantial factors drive a purchase?
• What problems are they trying to solve when they make a purchase? (What “job” are they hiring the product to do?)
• What else have they tried and what about those solutions did/didn’t work?
• What are their fears, hesitations, and concerns around making a purchase?

Once you know those things, you have a better idea not only of what to build, but how to market, sell, and improve something (more on that later).

Of course, if you’ve heard some of this before, you may be nodding your head — but no less stuck. You know you should do customer research (kinda like you know you should eat well, sleep, and exercise), but that doesn’t mean you do it.

Maybe because the how to research piece is about as clear as those foreign language assembly instructions you got from Amazon.

Let’s clear things up in plain English.

A near-comprehensive list of qualitative customer research methods

There’s a pretty wide range of customer research options. Which one is “best” for you will depend on your time, budget, and knowledge gaps (e.g. what question you’re trying to answer).

For example, a founder who’s worked in the cyber security niche and wants to build a better picture of customers has a different problem from a two-year-old cyber security company struggling with heavy churn.

The first will probably lean more heavily on the passive research options below vs. the second, which would benefit from 1:1 customer interviews with new sign-ups and recently churned customers.

But I’m getting ahead of myself a bit there.

Let’s look at your research options first, then tackle where to start.

Free, easy, and (mostly) low-effort research options

Best for:

• Getting the “lay of the land” for an audience
• Layering qualitative data on top of focused surveys and interviews

Main Methods:

• Niche communities: Look for Slack groups, Facebook communities, Twist groups, and other communities (including local!) that your audience gravitates to. Then, pay attention to common frustrations, excitements, and anything else that defines your group.
• Forums: Quora, Reddit, HackerNews...there’s a forum for most audiences. Find the one for your audience (or find the questions your audience is asking on common forums like Quora) and spend some time lurking and listening.
• Review sites: Depending on the audience, you may find some great qualitative data on review sites like G2 or Capterra. What do your customers love and hate about other products they use?
• Competitor sites: If review sites aren’t an option, or if you want to hone in on one particular competitor, check out their site. If they’re smart, they’ll display a lot of social proof, which will help you figure out what customers love about them from the customers’ perspective.                            

• Social media: From threads to hashtags to search to communities, there are loads of ways to learn more about most audiences through social.
• Amazon review mining: That’s not a typo. Here’s how this can look: one time I was researching why a target audience would consider a paid course on SEO. So, in addition to the methods above, I looked up Amazon reviews of books on that topic, and they helped me see where the book alternative fell short and what the audience was expecting from it.
• Sparktoro: A godsend of a tool, SparkToro helps you quickly identify important websites, accounts, and other influential platforms for your audience. (Fun fact: Rand Fishkin built this platform on, you guessed it, a lot of customer research.)   

Keep in mind: As you’re observing and gathering notes, you’ll need to read between the lines with this kind of research. Alex Hillman (co-founder of 30x500 and author of Tiny MBA) reminded me that what an audience says they’re struggling with may not be what they’re actually struggling with.

So, treat insights like clues. Alex recommends, “interrogate the clues and say, ‘Why is this here in the first place for this population?’...You have to look at the environment. Go a little Scooby-Doo on the situation. Investigate, look for more clues.” This takes effort, but Alex says it’s “the work that your competition will never do to really understand your customer” So, it’s well worth your time and effort.

Low-cost and moderate-effort research options

Best for:

• Answering specific audience questions
• Gathering a large volume of research

Methods:

• Surveys: Emailed, on-site, or in-app, there are loads of ways you can run surveys. If you’re trying to gather a large amount of representative data, perhaps you email out a few questions, the way Hiten Shah did while building FYI. Or, if you’re trying to answer one very specific question, maybe you take Amy Hoy’s approach below to nail down who’s coming to your site. Check out Hubspot for guidance on doing any option well.                       

• Chat transcripts/data: If site visitors are engaging with your chatbot, there are guaranteed some nuggets of goodness in there. Read through what customers are asking or searching for to get a better feel for their hopes, intentions, frustrations, and hesitations.
• Sales or demo call transcripts: If you have an existing product that’s requiring personal touchpoints for a sale, browse those transcripts. What questions are customers asking on sales calls or in demos? What hesitations or concerns are they expressing?
• Customer service tickets: People who take initiative to open up support requests are generally either very happy or very upset about something. Both types of information are useful. Not to mention, this helps you get a good feel for how customers talk (Casual? Formal? Emojis?)

Keep in mind: Surveys and transcripts are great for gathering quantitative data or getting answers to very specific questions. But you’ll often want to layer on 1:1 customer research for a richer picture. That’s because, except for surveys, the options above aren’t geared toward in-depth research. They’re useful for sure, but they take place in the context of making a sale or resolving support — the goal isn’t to learn, it’s to sell or resolve. For the most in-depth learning opportunities, you’ll want to do customer interviews.

High effort, high impact research options

Best for:

• Gathering rich, detailed insights from a small group of customers

Methods: The main method here is 1:1 customer interviews. You reach out to a small set of customers, set up a time to chat, interview them, and listen carefully. (Ideally, you do this routinely.)

Keep in mind: You’re not asking, “what do you want?” in interviews. You’re looking for stories, specifics, and rich details.                                   

And, not to be a Debbie Downer, there are a lot of ways to screw this one up. You could interview the wrong customer segment, pollute the interview with loads of leading questions, or talk more than you listen. I’m not pointing this out to discourage you — I’m mentioning it to encourage you not to start from scratch!

There are a lot of great resources out there on this topic. For detailed help selecting who to interview and exactly what to ask them, check these out:

Free

• Baremetrics guide to customer research (article)
• Customer Interview Questions (free resource by Stuart Balcombe)
• How to interview your customers by Justin Wilcox (article)
• Our list of 7 things NOT to do in customer interviews

Paid (but worth it)

• The Mom Test by Rob Fitzpatrick (book)
• Clarity Call Cheatsheets by Customer Camp (templates that take you from choosing who to interview to making sense of the interviews; I own these and they’re excellent)

So, where do you start? With a Big Hairy Question

Like most business advice, a “one size fits all” approach rarely fits. Interviews aren’t always the best options — likewise for surveys and online scouting.

Ideally, you want to start with a Big Hairy Question you’re facing and then use a variety of research methods to uncover answers.

Take the example we mentioned earlier: you have a two-year-old cyber security product and churn rates are keeping you up at night. Your Big Hairy Question is: what factors cause existing customers to churn?

To answer that, you could use a few different research methods and look for patterns:

• Survey recent signups. Email recent signups as a part of onboarding or use an in-app survey to figure out, “why did you sign up for this?” Understanding what they expect from your product, or what job they hired it to do, could help you figure out if you’re falling short of their expectations (or selling one thing and delivering another).
• Survey recent churns. Ask folks why they left. Track their responses (more on that below) and see if they cite similar reasons or types of reasons. Do they need your product for a specific, short-term problem? Do they change jobs/roles frequently? Were they expecting something different?
• Interview recent churns. This is probably where your richest insights lie. Set up quick interviews with recent cancellations and ask them why they canceled.
• Interview recent signups. Go deep into why customers purchased and what they expect. Compare these notes to the reasons customers are churning. Are there any relationships or patterns?
• Read testimonials from competitors. Do some passive online research and see what makes customers stick with your biggest competitors. Most companies amass social proof all over their landing pages (for good reason — it boosts conversions!). Use that to your advantage. What do their customers rave about?

Doing a few of these — especially interviewing recent churns — is a good start. Doing all of them will help you paint a detailed picture and understand the why behind those concerning churn numbers.

*Note: This article primarily focuses on qualitative research methods. But keep in mind you can also find quantitative customer information in Google Analytics, heatmaps (check out Hotjar), and product usage stats. Those are worth diving into — especially if you have substantial site traffic or product usage.

You WILL find insights — now, here’s what to do with them

From doing customer research ourselves, we can tell you we’ve never regretted it. And we always learn something.

But once you round up some insights, you’re going to face a potential pitfall. You could learn something and stash it on the shelf. And sadly, that’s what many companies do. It’s not that they mean to waste what they’ve learned — more often, it’s they don’t know how to put it to good use.

To sidestep this pitfall (a shelved insight doesn’t help anyone or any business goals!), make sure you:

• Gather: Record the key insights you learn
• Analyze: Map those insights to the customer/buyer journey
• Leverage: USE that map to improve product, sales, marketing, and more

Here are each of those steps in more detail.

GATHER: Record and document what you learn

Unfortunately, there’s no easy, affordable way to store and organize all research findings (that we know of). But that doesn’t mean your findings have to live in a hundred sticky notes or text documents on your desk.

Here are a few different ideas for keeping things organized:

Must do:

• Record and transcribe all interviews and conversations. Tools like otter and Rev (check out the rough draft option for 25 cents/minute) are great options for transcribing.
• Store transcripts in a central location. Keep a shared folder of interview notes/transcriptions somewhere accessible — Drive, Dropbox, whatever you use.

Could do:

• Create a #customer_insights Slack channel. We do this, and it helps us keep track of important insights and conversations.

Create an insights repository. Use one of your existing tools (Trello, Airtable, Notion, whatever) to store, tag, and organize insights from various research methods. A living document like this takes work, but it helps team members search and find insights relevant to their question.                                               

I've used airtable to organize research, and it's a great way to keep takeaways in one centralized place.                                

Documenting what you learn will help you with a very important next step: analyzing your findings.

ANALYZE: Find patterns or themes and map to the customer journey

As you amass your research, you want to start pinpointing clues and finding patterns. There are a few useful frameworks for this, but my favorite is the customer journey map.

There are several ways a journey map can look, but the basic idea is this: it outlines how a customer gets to and builds a relationship with you from the customer’s perspective.                           

Note: Most companies approach this from the flip side: they map out funnels, from a business perspective, of how customers interact with the product from “what’s this?” to “renew subscription.” Those funnels have a place internally, but the goal of research is to understand the customer’s perspective, so they’re not your best tool for analyzing research.

To build your own map, the first thing you want to do is identify the key steps a customer takes in their journey. Most journeys include some or all of the points below:

• Trigger: Things that make a customer realize they have a pain/need/problem.
• Searching: How a customer goes about trying to find a solution.
• Testing: The DIY, free, or accessible things they initially try.
• Deciding: The rational and irrational steps they take from testing alternatives to choosing your solution.
• Purchasing: The experience they have buying, receiving, and engaging with your product or service for the first time.
• Evaluating: Their first few days or weeks with you
• Renewal/upgrade: Taking their relationship to “the next level”
• Loyal advocate: Becoming a true fan and promoter
• Abandonment: Deciding your solution doesn’t meet their needs and stepping away.

You’ll probably refine whatever journey you start with over time, and that’s great. But as soon as you have that first version, you want to start mapping your existing research to each of those points. This includes doing things like:

• Associating actual word-for-word quotes to each stage. “I tried everything — self-help books, online courses, a business coach...none of it helped.”
• Identifying patterns of thinking and feeling at each crossroad (common emotions and sentiments). “$99/month is a BIG commitment for my business. I was worried it wouldn’t be worth it.”
• Pinpointing the catalysts that move customers forward or backward in their journey. “We first started looking for a new mattress when we got a dog and it wouldn’t sleep anywhere but in our bed.”
• Noting opportunities to help, reassure, or reduce friction for customers along the way (less fraction = faster journey)

From there, it’s all about putting that map to good use.

LEVERAGE: Use your map to improve product, sales, marketing, and more

Because the customer journey touches every part of your business, a detailed map of is something every area of your brand can benefit from.

Don’t believe me?

Here are just a handful of applications:

• Channel prioritization: Figuring out where your customers hang out helps you narrow down how to reach them.
• Traffic generation: Learning what triggers a customer to start looking for you gives you a better idea what ad copy or SEO queries to target.
• Conversions: Knowing your customers’ pain points, hopes, etc will help you craft and test more effective product and landing pages. (It once helped me write a landing page that generated $10k+ in 24 hours.)
• Positioning: We’re written before about how this can make or break your business. Figuring out what your best customers hire your product to do helps you narrow down how to pitch it to other customers like them.
• Churn: Knowing why customers leave gives you tangible ideas on how you can address churn and better educate/equip future signups to keep them around.
• Product roadmap: Figure out what customers want from your product and you’ll have a better idea which features and improvements to prioritize.

Set out to answer your Big Hairy Question, but keep track of other opportunities and applications you come across — trust me, there will be plenty!

Too good to be true? Common roadblocks in real-world customer research (and how to get around them)

Some of my fellow cynics may be skeptical of how good customer research sounds right about now. Surely if this is common knowledge, more founders and companies would be doing it, right?

The fact is, the ones who know this stuff do start doing research...but many have a hard time maintaining it.

And we know from doing customer research why some companies fall off the wagon — or never jump on to begin with.

Customer research is usually pitched like this: research → analyze → profit.

But like most journeys worth taking, it’s more complex than that. There are plenty of road bumps (or roadblocks) along the way. Boulders you have to climb over, rivers you have to fjord, worn-out shoes you need to replace...you get the idea.

And while there are challenges, I promise this isn’t a rigged version of Oregon Trail (“you have died of drowning in surveys”). There are legitimate ways around the roadblocks you have.

Here are some notable blocks we’ve come across and ideas for getting around them:

Uncertainty around the Big Hairy Question

The issue: The first common roadblock is not knowing your Big Hairy Question. This sounds harmless (“can’t you just...pick a direction?”), but it makes research quite difficult. Without a clear question, it’s hard to know who to research, what to look for/ask them, and what to do with the data you find.                              

How to get around this: Before you use any research method, write out what the heck you’re trying to do. Understand top pain points for an audience? Figure out what jobs an audience hires a product type to do? Understand what factors motivate a customer to choose you? Whatever it is, write it down.

Other priorities compete for time, budget, and resources

The issue: Profitwell found 7 out of 10 organizations are speaking to less than 10 prospects or customers per month. I’m betting at least some of this comes down to competing priorities. When the pain of leaking revenue from dry sales, high churn, or low signups feels acute, it’s tough to prioritize research.

How to get around this: Prioritizing research is something you do in stages:

• Short-term, make the business case for customer research (especially if you’re not in charge of budget). Say you take one week to interview customers and figure out what factors are causing them to churn. If you then know what to fix, how much time and money could this save your business over guessing what could help?

• Mid-term, start with free research options you can sneak in alongside other priorities. Apply your findings, track results, and keep building that business case.
• Long-term, promote a culture that’s customer-centric and runs on knowing and serving the customer. In that environment, customer research isn’t debated — it’s non-negotiable.

Stalling out in the analysis stage

The issue: You gather all this great information...and, uh, now what?

How to get around this: There’s no silver bullet here. A lot of it comes down to two things:

• Consistently documenting what you learn
• Setting aside time to review learning.

To tackle the first, set up some kind of system like the one we described earlier. To deal with the second, consider setting up a recurring meeting on your calendar (with yourself or key team members) to review what you’re learning each week or month. For example, Calendly uses a weekly process where they collect nearly every piece of customer input and then triage to zero.

Believing you’ve “uncovered all the good stuff”

The issue: After a month or so, you’ll have some really good stuff. And you might start to think you’ve gotten the most bang for your buck. Time to move on to other stuff, right? In reality, it’s probably time to move onto another question.

The market and customers are always changing. If you don’t stay on top of research, you’re going to fall behind.

How to get around this: Make customer research a habit. So much so that it’s harder to stop doing it than to start doing it. What this could look like:

• Interview five customers a month
• Read through chat, sales, or support transcripts 1x/wk
• Set up quarterly meetings to review key research findings with your team

Hiten Shah has built and advised plenty of successful companies. Yet he started researching before he built FYI. Then he kept doing research when he launched an MVP and beta. And he’s not stopping anytime soon now that the product is live.

There’s something to learn from Shah’s approach there. Do whatever you need to start building a culture of research where customer understanding isn’t optional, it’s how you do business.