Knowing Who You Are Gathering Insights From

By Aileen Wu


One of my favorite activities related to product development is gathering consumer insights. I enjoy not only the process of observing people and interpreting their feelings and motivations, but also sitting at the front lines of where your creation meets the test of the outside world. Throughout Launching Tech Ventures and entrepreneurial endeavors, I have seen and used many techniques to gather insights from smoke tests to surveys to usability tests. In order to get reliable and valid customer data, it is important to think about not only what test to use, but also who to perform the tests on.

Dharmesh Shah highlights the dangers of acting on survey data in a great article called “Early Evidence Is Often Too Early And Not Really Evidence.” He warns that even though 80% of your survey respondents might care tremendously about keeping an extra feature of your product, they may not be represent of your average user. Early on, you may have actually only gathered the responses of the few users of your entire user population who use this feature regularly.  This example brings to light the importance of collecting data from a randomized group of users.

When it comes to usability tests, it can be tricky to determine what type of user to watch. I went to great talk by UI designer, Juhan Sohin, who claimed it was important to design for the expert user, not the beginner. He cited the example of the wheel button on the first iPods. First time users fumbled when first introduced to the wheel because it was like nothing they had ever seen on a consumer electronics device before. However, after they discovered how the wheel worked, they embraced its clever design and ease of use.

When determining who to observe in usability tests, it is important to think about what the dominant use cases will be. If you are developing a product that users will only interact with occasionally, such as Comcast’s bill payer, it may be more effective to optimize website design for a beginner user. However, if you are developing a product like RentJuice that your target population, in this case real estate brokers, might be using for several hours each day, it may be more advantageous to observe an expert user (someone who has spent several hours using your website already).   

At the end of the day, it is important for you to take extra steps to know exactly who you are gathering insights from. You can use qualifying questions as well as randomization to make sure you are pulling data from a relatively unbiased group of users. Furthermore, it is important to think what type of user you want to help you make decisions to drive product design forward.



Comments

Post a Comment

Popular posts from this blog

Managing Your Online Persona, Part I

by Brandon Giles (republished from his personal blog ) Why Care? It is becoming more important than ever to actively manage your persona on the internet, particularly if you operate in the fields of entrepreneurship, venture capital or marketing. In general, persona management is more crucial in industries where your value and success depends on personal human capital and others perception of that value. However, digital footprints have a material impact on everyone. Beginning back in 2007, an ExecuNet survey  found that 83% of recruiters already used internet searches to evaluate job candidates, with 43% of recruiters eliminating candidates because of what they found. “How-to” Guide As an aspiring entrepreneur I sought to learn more about the tools, platforms and best practices before busting out onto the Blogging scene. I heard from nearly every guest in my Launching Tech Ventures entrepreneurship course at the Harvard Business School (#hbsltv) that blogging was critical to estab...
Read more

Letting it Go

by Anthemos Georgiades There are various lessons I could have discussed here regarding the UX tests I ran for my startup Zumper. But, instead, I am going to focus on one challenge that these tests created for me: my ability – or lack thereof – to let my own opinions and intuitions drop. In the three lean tests that I ran, two of them confirmed something I had suspected. The third one however did not. In my ‘banner’ test, I tested (1) the price users thought a standard bed frame was worth, under three different themes of page banners and then (2) if the user could actually remember the site’s name after use. I found that what I considered to be the most ‘boring’ white and grey banner drove the highest price estimations from users and beat our other two banners hands-down on brand recollection. I had actually only included this ‘boring’ banner in the lean test as a control. My hypothesis had been that our green banner would drive the best engagement given that this was the predominan...
Read more

Creating the “Next Big Thing": How to Inspire Innovation

By Brad Bonnett  How do I apply all my learning’s about lean startups such as finding product market fit with a minimum viable product if I don’t have an initial problem that I am trying to solve?  I know that I want to build something that changes the world but I continue to struggle to find what product or service I want to build.  During our class with Fred Wilson he said that “If you have an idea that you can't get out of your head, do a startup. Otherwise join a startup.”  Although I appreciate the simplicity of this view and the experience that Fred brings to the table I am dismayed to think that unless I stumble upon an idea that really sticks with me I am going to be excluded from starting my own company.  Therefore I tried to put together a process that can help inspire innovation for those of us who are lacking an idea to create that “next big thing!” I tried to take the best practices that I have learned from my Managing Innovation class at HBS and ...
Read more