Lessons in Early Demand Discovery

by Jason Lorentzen

My work focused on vetting potential demand for a web/mobile-based application that could be used to help people better understand the nutrition content of restaurant dishes. My approach was to use a sizable survey (link to survey) and user interviews to understand:

• people’s behavior as it related to restaurant selection 

• whether or not people cared about nutrition 

• how satisfied / dissatisfied users were with the lack of nutrition data at restaurants 

• differences between different demographic characteristics 

All in all, I received some good feedback (see survey results) that helped me determine not to go forward with my area of investigation. But my process was far from perfect. Below are my reflections on what I learned, what I wish I had learned, and how if I did it over again, I’d take a different approach.

What a Large Survey and Complementary Interviews Accomplished Well

Get a large set of data on general behaviors, cheaply. In relatively short order, my survey returned over 100 responses. Over 90% of respondents completed the entire survey, which had at least 29 questions. I believe respondents put up with the longer survey length because the questions were broken into easily-completed multiple-choice fragments, meaning the velocity at which people could go through the questions was high. As a result, there was no noticeable trend of decreasing response rates as people got toward the end of the survey.

Establish (lack of) compelling awareness of a problem. A key goal of my investigation was to uncover whether or not there was strong discontent amongst a niche in the dining community about the level of nutrition information available. By asking for satisfaction rankings, I tried to give people the opportunity to vet their anger with the current situation. They didn’t, and I learned people don’t see the problem the way I thought they would.

Understand what people use today. In both surveys and interviews, leaving room for open-ended responses for people to explain the ways they currently act / solved the problem led to the discovery of unanticipated behaviors and competitive products.

Find out whether or not people are like you. Like many would-be entrepreneurs, the product I had in mind was based in no small part on my own experience. Rather than simply relying on interviews, I got to understand attitudes across a large group of people, which helped me understand where people did and did not generally see the world like I did. Had I relied on interviews, it would have been hard to get the sample size necessary to establish a trend.

Lead generation. People who are interested in what you’re asking about / developing might leave their contact info and eventually turn into users. I happen to know many of the people who left their contact information on my survey, but some I don’t. Had I moved to a second phase, I would have a built-in base of alpha test users or focus group participants. 

What I Could Have Done Better

Force appropriate focus. My project could have benefitted from greater focus. It is tempting during investigative interviews and surveys to examine many different angles of a potential problem. Indeed, people have many different perspectives / experiences to inform a potential product. While I think my one-size-fits-all survey and open interviews allowed me to get a broad set of product ideas and behavioral insights, I think it also hindered my ability to focus on a single product concept. Without depth of understanding in any given concept/niche, I ended up with less useful information for product pivots. I also ended up with a lower number of respondents in any given niche, reducing the reliability of the conclusions I could draw.

Get people at the right time. I noticed in interviews, and I suspect this holds true for survey respondents, that my subjects were at times struggling to put think through the process of interest. For instance, when I wanted them to think about things that were important to them when ordering at a restaurant, they would pause to recall their behaviors. What might have been left out in the recall? I wish now I had gone with them to a restaurant to actually talk through what they were thinking as they were experiencing it.

Actually kill any idea. I may not have found a compelling dissatisfaction as a source of demand for a product, but I also don’t know if people would have, once they saw/used a product, been excited by it. 

 

If I Did It Again…

After going through this survey and interview development process, I’d do a few things differently. I think the following are useful tips for those who are very early in the process of vetting an idea and are trying to get a sense of demand.

1. Target one specific niche/concept at a time and target questions specifically to them. By being focused, you can ask fewer questions but get ultimately deeper, more meaningful insights about what that specific customer wants or needs. These deep learnings can help you discover areas of untapped opportunity. And if one niche proves undesirable, you can iterate into the next one.
2. Build a (crude) prototype before you survey / interview, in order to focus your own thinking and show it to respondents to get more powerful/specific reactions. You can (and should) still ask behavioral questions before you show product concepts if you want to get unbiased data on attitudes.
3. Cut the fat (unnecessary survey questions) and focus on what matters. Questions that directly vetted my hypothesis were useful. Questions that were more exploratory seemed interesting at first but didn’t have appropriate supporting data to be actionable.
4. But leave room for a few open-ended answers. Allow people to surprise you by helping you discover competing products or coping mechanisms that better illustrate their current behavior. And make sure to get their contact info if you can for follow-up (but don’t force them to leave it!).
5. Pay for survey targeting or find another way to get outside the bubble. Another advantage of a shorter, more focused survey is that people that don’t know you well might be willing to take it. Passionate early adopters might even be excited to look at concepts. Having an HBS-centered response group will taint your own faith in the results.
6. Interview people in the moment. People’s impressions will be most accurate when they’re engaging in the activity / process that you want to improve.