Learnings from using customer surveys and expert interviews to build a customer profile
by Bryan O’Connell
For my project, I used a combination of customer surveys and expert interviews to build up a profile of potential customers for my business (and indeed to stress test whether the idea was of value). I learned a number of important lessons on how to gather effective survey data, many of which seem incredibly simple and obvious. However, from looking at other surveys I have received recently, they are seldom followed.
1) Keep it short
For any business plan, there is probably a small number of datapoints that provide huge insight into its viability. For this business, it was whether or not teachers actually already vacation outside North America, as if they do, this would imply that our business would cannibalize existing sales and would not be viable. Information on where teachers vacation is not publicly available, so the only way to gather these data was by means of a survey.
Once you have convinced a respondent to open your survey, it is incredibly tempting to ask them to answer many non-core questions, each of which might add only a fraction of the value of your “key questions”. However, if you reduce your sample size by even 1, then it probably wasn’t worth it. This is why I keep my surveys to a maximum of 10 questions that fit on one page and are easy to answer (i.e. limited amount of free text). From talking to classmates, I believe this was one of the critical factors in ensuring a high response rate.
2) Keep it simple
Remember, just because someone has opened your email doesn’t mean they will do everything you tell them to. I once saw a survey that asked people to time themselves performing certain tasks on a website, while simultaneously counting clicks, pages and other metrics. I tried to help and be as diligent as possible, but I quickly lost count and my results were meaningless. I think it’s critical to remember that some tasks need to be performed in live focus groups and are just not conducive to surveys.
3) Have someone else double check your survey
When you are incredibly close to the project, everything seems obvious to you.
However, the subtlest ambiguity can render entire datasets meaningless.
Also its important to get the simple things right. For example, in multiple choice surveys when dealing with numbers, rank answers in order. I recently received a survey with the following answer choices:
4) Use your network, but don’t let it make you lazy
The HBS network is a fantastic resource - the combination of ready access to listservs, and people that are more than willing to be helpful, means that we can very quickly and easily get a large sample size of people to answer any survey. However, its important to realize that HBS people are not representative of the general population, and should only really be relied upon when the real life population we are interested in is a close approximation of the HBS demographic. With a little more work however, the HBS community (or indeed any similar professional network) can be used to give us access to pretty much any perspective.
For my project, I wanted to get a sample of teachers living outside major cities in the United States. As a foreigner, this would have been very difficult using my personal network. However, by drawing on the HBS network I was able to get every key question I wanted answered by a large sample (50 teachers).
5) Interviews can be a useful supplement to surveys, but remember their statistical significance
Lastly, interviews are great for providing more color, but remember that they are just one datapoint. Remember not to add extra weight to what an interviewee is saying just because you are in their presence and they can put their point across more forcefully.
For my project, I used a combination of customer surveys and expert interviews to build up a profile of potential customers for my business (and indeed to stress test whether the idea was of value). I learned a number of important lessons on how to gather effective survey data, many of which seem incredibly simple and obvious. However, from looking at other surveys I have received recently, they are seldom followed.
1) Keep it short
For any business plan, there is probably a small number of datapoints that provide huge insight into its viability. For this business, it was whether or not teachers actually already vacation outside North America, as if they do, this would imply that our business would cannibalize existing sales and would not be viable. Information on where teachers vacation is not publicly available, so the only way to gather these data was by means of a survey.
Once you have convinced a respondent to open your survey, it is incredibly tempting to ask them to answer many non-core questions, each of which might add only a fraction of the value of your “key questions”. However, if you reduce your sample size by even 1, then it probably wasn’t worth it. This is why I keep my surveys to a maximum of 10 questions that fit on one page and are easy to answer (i.e. limited amount of free text). From talking to classmates, I believe this was one of the critical factors in ensuring a high response rate.
2) Keep it simple
Remember, just because someone has opened your email doesn’t mean they will do everything you tell them to. I once saw a survey that asked people to time themselves performing certain tasks on a website, while simultaneously counting clicks, pages and other metrics. I tried to help and be as diligent as possible, but I quickly lost count and my results were meaningless. I think it’s critical to remember that some tasks need to be performed in live focus groups and are just not conducive to surveys.
3) Have someone else double check your survey
When you are incredibly close to the project, everything seems obvious to you.
However, the subtlest ambiguity can render entire datasets meaningless.
Also its important to get the simple things right. For example, in multiple choice surveys when dealing with numbers, rank answers in order. I recently received a survey with the following answer choices:
- Once a week
- Once a month
- Twice a month
- Once a quarter
- Once a year
4) Use your network, but don’t let it make you lazy
The HBS network is a fantastic resource - the combination of ready access to listservs, and people that are more than willing to be helpful, means that we can very quickly and easily get a large sample size of people to answer any survey. However, its important to realize that HBS people are not representative of the general population, and should only really be relied upon when the real life population we are interested in is a close approximation of the HBS demographic. With a little more work however, the HBS community (or indeed any similar professional network) can be used to give us access to pretty much any perspective.
For my project, I wanted to get a sample of teachers living outside major cities in the United States. As a foreigner, this would have been very difficult using my personal network. However, by drawing on the HBS network I was able to get every key question I wanted answered by a large sample (50 teachers).
5) Interviews can be a useful supplement to surveys, but remember their statistical significance
Lastly, interviews are great for providing more color, but remember that they are just one datapoint. Remember not to add extra weight to what an interviewee is saying just because you are in their presence and they can put their point across more forcefully.
Comments
Post a Comment