Customer Feedback Testing: A Quick Guide to When & What to Use
by Xing Wang
In lean startups, customer feedback is vitally important because it informs product development. However, customer feedback is equally as important on the customer discovery side as it is for product development: for determining the customer value proposition, likely demand, purchase behavior, and overall attributes of the target consumer.
Value Proposition: To validate if initial hypotheses on your product’s customer value proposition, surveys are an inexpensive but relatively reliable methodology to use. It is vitally important to define the right audience to send the survey to (are your friends really the right target market?) and also choose wording carefully in surveys. The structure and wording of questions in an online survey can skew results incredibly. For example, asking “Would you be interested in Product X? Yes or No” will lead to a very difference outcome than “pick the most interesting product from the list below.” Watch out not to mislead the consumer or else surveying becomes an unreliable benchmark to use!
Product Demand: Simple smoke tests are a cheap way to test product demand. In smoke tests, the product tested is not operational/functional, and instead users are responding with demand to a simple description or other mechanism that approximates the eventual product. Similar to what we see in the Aaadvark case, one can send users to a landing page to see if they would respond to an invitation, or, as in Triangulate, test out to see how many users would click on a dummy “photo” button.
Target Customer: To define a target customer, focus groups or one-on-one interviews are powerful tools through which an entrepreneur can understand the consumer psyche, particularly if he or she is still trying to define who to target as a customer. This technique should be used carefully. While qualitative research is incredibly informative, it can be misleading because customers do not always behave in reality the way that they indicate in focus groups. What one “takes away” from a focus group is also incredibly subjective and vulnerable to biases or multiple interpretations. As we see in the Cake Financial case, timing on when in the life cycle of your startup to rely on focus groups is also very important – focusing too much on what the customer is saying in the very beginning is another potential trap.
Customer Behavior: Some of the best tests for how a customer will interact are through usability tests. By observing what customers are doing, an entrepreneur can quickly pinpoint potential pain points in the customer experience with the product. Drew Houston in Dropbox uses this to great effect when he was able to learn from watching recruits from Craigslist struggle at how to use the Dropbox product. These tests are particularly useful because unlike other types of qualitative research where the customer TELLS you what he or she thinks, you are able to observe actual behaviors, giving a more accurate read on what is going on.
Product Development: Customer feedback on the product development side is extremely tricky to handle. While “the customer is always king”, startups must beware that customers don’t always REALLY know what they want. In particular, in tests like focus groups or one-on-one interviews, what the customer indicates as their wants in that particular setting and moment is not necessarily reflective of how they will actually act in the spur of the moment. This is especially the case with B2C products where a purchase decision is heavily driven by emotions of the moment, i.e. a fashion startup. In fact, as we see in the case of Dropbox, oftentimes what the consumer wants may not align with the ultimate vision of the founder or what the founder believes to be best for the company or product (i.e. syncing Dropbox with My documents). There is a fundamental balance that a founder must carefully consider between user-centric design and top-down product vision.
In lean startups, customer feedback is vitally important because it informs product development. However, customer feedback is equally as important on the customer discovery side as it is for product development: for determining the customer value proposition, likely demand, purchase behavior, and overall attributes of the target consumer.
Value Proposition: To validate if initial hypotheses on your product’s customer value proposition, surveys are an inexpensive but relatively reliable methodology to use. It is vitally important to define the right audience to send the survey to (are your friends really the right target market?) and also choose wording carefully in surveys. The structure and wording of questions in an online survey can skew results incredibly. For example, asking “Would you be interested in Product X? Yes or No” will lead to a very difference outcome than “pick the most interesting product from the list below.” Watch out not to mislead the consumer or else surveying becomes an unreliable benchmark to use!
Product Demand: Simple smoke tests are a cheap way to test product demand. In smoke tests, the product tested is not operational/functional, and instead users are responding with demand to a simple description or other mechanism that approximates the eventual product. Similar to what we see in the Aaadvark case, one can send users to a landing page to see if they would respond to an invitation, or, as in Triangulate, test out to see how many users would click on a dummy “photo” button.
Target Customer: To define a target customer, focus groups or one-on-one interviews are powerful tools through which an entrepreneur can understand the consumer psyche, particularly if he or she is still trying to define who to target as a customer. This technique should be used carefully. While qualitative research is incredibly informative, it can be misleading because customers do not always behave in reality the way that they indicate in focus groups. What one “takes away” from a focus group is also incredibly subjective and vulnerable to biases or multiple interpretations. As we see in the Cake Financial case, timing on when in the life cycle of your startup to rely on focus groups is also very important – focusing too much on what the customer is saying in the very beginning is another potential trap.
Customer Behavior: Some of the best tests for how a customer will interact are through usability tests. By observing what customers are doing, an entrepreneur can quickly pinpoint potential pain points in the customer experience with the product. Drew Houston in Dropbox uses this to great effect when he was able to learn from watching recruits from Craigslist struggle at how to use the Dropbox product. These tests are particularly useful because unlike other types of qualitative research where the customer TELLS you what he or she thinks, you are able to observe actual behaviors, giving a more accurate read on what is going on.
Product Development: Customer feedback on the product development side is extremely tricky to handle. While “the customer is always king”, startups must beware that customers don’t always REALLY know what they want. In particular, in tests like focus groups or one-on-one interviews, what the customer indicates as their wants in that particular setting and moment is not necessarily reflective of how they will actually act in the spur of the moment. This is especially the case with B2C products where a purchase decision is heavily driven by emotions of the moment, i.e. a fashion startup. In fact, as we see in the case of Dropbox, oftentimes what the consumer wants may not align with the ultimate vision of the founder or what the founder believes to be best for the company or product (i.e. syncing Dropbox with My documents). There is a fundamental balance that a founder must carefully consider between user-centric design and top-down product vision.
Comments
Post a Comment