Archive for January, 2010
Tags: contextual interviews, Customer research, detailed interviews, ethnography, personas
This is the fifth post in my Customer Research series.
There are many awesome posts on personas, including this primer by Steve Johnson at Pragmatic Marketing, and this post about how personas serve the CEO and the executive team. So I am not going to belabor the point. I’ll simply provide a case study of exactly how I go about building personas.
Before we begin, we must decide whether we are building buyer personas or user personas. Since I have always worked on the PD/PM side (instead of the PMM side), I almost always end up making user personas first. Here is the process I use.
1. Pick a market segment. Size it. Make sure the total addressable market (TAM) is big enough to matter.
2. Generate prototype personas. Draw on your company’s tribal knowledge / theories about customers and prospects, as well as from your own life experiences for this. Make sure you state assumptions about needs, wants and expectations.
3. Create a screening questionnaire. Work with a recruiter to find subjects. Evolve screener as necessary (tweaking the prototype personas as needed).
4. Plan the interviews.
- Pull together the equipment: a mini DVD camcorder, a tripod, fully charged batteries, camera, laptop for notetaking. (Mini-DVD camcorders result in the least postprocessing overhead. Finalize the disk and you can watch it instantly.)
- Staff each interview with 1 researcher who will lead the conversation, and at least 1 support person (in the role of audio/video monkey and notetaker).
- Every researcher should plan on going to several interviews (overlapping with each other for some of them).
- Cycle as many staff members through the support role as possible. This exposes everyone to the process and gets people out of the office and into customers’ environments, which is always good.
- Plan mid term and final debrief sessions with the research team.
- Plan other work around this work. It takes at least 1 – 1.5 person days to process each interview. You also need time to reflect on the results. Expect each researcher to be completely consumed for a good 75% of the time.
- Share intermediate work products with the team early and often. Transparency is key to buy-in.
5. Do the interviews. Here are some tips I’ve collected over the past 14 years (your mileage may vary).
- For best results, detailed interviews should be done in the target environment for product use. That said, an interview at a coffee shop or some other neutral environment is better than no interview at all.
- Start with an introduction and a warm up section where the subject gets comfortable with the researchers, then ease into the topic of interest.
- Keep the interview guide short and sweet. Questions should be open ended, inviting the interviewee to talk in their own style and to show their personalities.
- Be prepared to take the conversation where it wants to go, not where you want it to go.
- Make sure the subject talks more than you.
- For certain types of products, you may have to match gender. (e.g. research around a feminine hygiene product will require a female interviewer).
- Don’t even think about showing a product or product concepts. Discuss your product at the end only if there’s time left over.
6. Build the personas. With luck, you will have recruited carefully to get the right people, and after 10-20 interviews, they will have self-organized. You should now be able to build composite personas in each bucket of subjects who share key needs, wants, expectations, and attributes. Check these personas against the people you met and tweak until they work well.
Once the primary, secondary and non-personas are built, you can begin to use them as a proxy to help you envision what benefits they require, what products or services can deliver these benefits, and how those products and services may look like.
Tags: Customer research, detailed interviews, ethnography, immersion, observation, persona, shadowing
This is the fourth post in my Customer Research series.
The word “ethnography” has such a grand sound to it. I’ve seen seasoned executives get intimidated by it. They say things like “Whoa – that’s big agency stuff! We don’t have budget for that!” To that, my standard response is: “It’s not rocket science! We can do this!”
And it really isn’t rocket science. It’s just hard, detailed work. Lots of it. You don’t need a degree in anthropology or psychology. Anyone with an open mind and great listening skills can learn to do it. If you can’t listen more than you talk… well, perhaps you shouldn’t be a product person after all.
And the budget can vary wildly. You can spend $250,000 if you outsource it to a big agency and you have a global market. You can train your staff to do it themselves under some guidance for well under $25,000, using a recruiter to find subjects for you. I’ve run projects under $2500 where I did the recruitment myself using Craig’s List. So your mileage may vary.
Now what exactly is ethnographical research in the context of product development? In the simplest terms, it’s a set of qualitative techniques that places the researcher in the environment and/or the mindset of the subjects. One listens and observes subjects in the environment that the product is meant to be used in, without showing the product or product concepts. One then derives the needs, wants, expectations and workarounds for the subjects, and uses this information to drive product definition.
There are three techniques that I am a big fan of (mainly due to their high bang for the buck):
- Detailed Interviews. Researchers meet with a subject for one to one and a half hours. A researcher asks open ended questions to get the subject to tell a story about the domain of interest. The interview guide tends to be very high level and the researcher is trained to mix things up and respond to new threads that come up in conversation. I like to have two to three researchers at each interview so one person can drive the conversation while the other person mans the audio/video equipment and takes notes.
- Observation or shadowing. Researchers set up their audio/video equipment in the environment where a product or set of products is being used, and simply hang out with the subject while the subject uses the product. The researchers asks questions when necessary, but by and large they behave like flies on the wall. They are there to watch and learn, not to talk. The subject is disturbed as little as possible. This could be a multi-hour proposition – I once shadowed someone for 8h with a coworker.
- Immersion. Researchers use the product or a related product for an extended period of time to get a first hand understanding of long term product use. Researchers can record their extended findings via a journal or a photo essay. I also like to write a debrief document at the end of the immersion to sum up my key takeaways.
There are other techniques too, based more on self-reporting by end users. Examples include journal studies or photo essays involving customers. These are all valid approaches. When used with detailed interviews and/or observation, they can help round out the picture of the customer’s problems, needs, wants, expectations and so on.
The biggest challenge for this technique is the amount of time it takes to do a good job. Since research is done one customer at a time, and best practices suggest we work with 10-20 customers to allow the data to converge and allow time for tweaking the technique and/or the recruitment criteria, it takes corporate level commitment to pull off a project like this. It’s a lot of long nights too since sometimes the research must be done after hours when the customers have time to work with the researchers. But if we keep the focus, and we recruit correctly, generally by the 5th or 6th interview or observation session, a pattern will begin to emerge.
You know you’ve nailed it when you start hearing the same thing from everybody over and over again. It’s an incredible feeling to get there and know you’ve built the knowledge that will guide the product team moving forward.
Tags: Customer research, customers, product discovery research, Product Management, product research, prospects
This is the third post in my Customer Research series.
One of the trickiest things about customer research is subject recruitment. The quality and applicability of research results is directly proportional to one’s ability to ask the right questions of the right people.
There are generally two types of customer research that directly impact product development:
- Product discovery research. This is what typically happens at the beginning of a new product development effort. The goal is to understand who we are developing products for, what their problems are and what solutions may serve their needs and meet their expectations.
- Product research. This is typically done after a product launch and the goal is to investigate product utility and usability and to gauge customer satisfaction.
Product discovery research should be done with prospective customers, not existing customers, for two reasons:
- Existing customers of a previous generation product may not fit the primary persona for the new product, so we may end up asking the right questions of the wrong people.
- Even if existing customers are smack in the sweet spot for the new product too, their expectations for the company’s value proposition have been set by the specific feature set offered by the previous product. Their needs may not be any different but their wants will be shaped by what they know about the previous product. We would be asking the wrong questions of the right people then – these folks are much better equipped to help with product research instead of product discovery research.
Product research, on the contrary, is best done with real customers and end users who would have much more to share about long term use of the product. One can do some product research with prospective customers too, but generally unless there is a desire to start selling the product to a completely different buyer/user persona than before, it is better to stick with existing customers.
That said, nothing is absolute in customer research. A certain amount of insight can always be generated by a bit of cross-recruiting. For instance, during product discovery research, one can test product features with a customer advisory board who can provide a rapid sniff test without imposing the overhead of recruiting new subjects. They can also comment on whether key features are missing in their mainstream workflos. During product research, one might want to test product usage with a completely new target demographic and see whether the product as it stands today can be expanded to serve new market segments.
In general I believe in being highly creative and mixing things up when doing research. Every product development project is different and when one looks at the facts at hand, it will be quite obvious which types of research efforts will yield the most bang for the buck.
Tags: Customer research, functional specifications, new product development, personas
This is the second post in my Customer Research series.
Many people have asked me how persona work leads to product definition. Here is my personal take.
In any new product development effort, we need to ask the following questions:
- What is the market segment I am shooting for, and is it big enough?
- Who am I making this product for?
- What are the problems that customers are trying to solve?
- How may our core competencies be applied to solve these problems?
- What are the product features that may deliver these solutions?
- What is the end user’s mainstream workflow, and what minimum feature set is necessary to service that workflow?
- How would the end user want these features to work for Version 1?
- What would the roadmap look like for future releases?
Item 1 is a market sizing and analysis exercise. It is based on quantitative data from analyst reports and from internal market analysis. It builds the business case for a new product, and helps establish a basis for interest that justifies further effort.
Items 2 and 3 are addressed by persona development. I like to interview prospective customers in the environment they will be using the product in about their thoughts on the problems that the future product may address. I listen between the lines and then derive the needs, wants and expectations of the prospective customers.
The key words are “prospective” and “expectations”. Expectations are set by experiences, which is why you will get a different result if you interview existing customers of a previous-generation product, as opposed to prospective customers that represent your target market.
Item 4 is a cross functional discussion involving the product manager, the engineering lead, and a proxy for an archetypical end user in the sweet spot, representing the primary persona we are trying to target. For instance, if the target user is a 65 year old, there should be an older person in the discussion to represent their point of view.
If the core competencies do not support a viable solution to the customers’ problems, the team would have to go back to Item 1 and rethink the target market and the business case. Developing new core competencies is an engineering research activity that precedes product development.
Item 5 is about envisioning features that will deliver the solutions needed by end users, and Item 6 places them in the context of user workflows. One is meaningless without the other.
At this point is generally a lively discussion with all the possible ways to serve up benefits to customers. Rather than going around in circles in the office, I reach out to customers to get their feedback. I employ a few techniques for this:
- Get customer feedback with paper prototypes. This can be done either as a round-table discussion (2-3 groups of 8-12 participants each) or remotely by phone on a one-on-one basis. This should be done with both prospective customers and existing customers so we can get the full range of perspectives for people in the sweet spot and for early adopters. Results should be interpreted separately, of course.
- Beta testing with hacked prototypes where feasible. This should run for 2-4 weeks with prospects AND existing customers with separated analysis of results for the two constituencies.
- A product concept test for the new product, executed as an on-line survey to prospective customers only.
- A comparative product concept test, pitting the proposed product against a control product, executed as an on-line survey to existing customers and prospective customers.
Once we get through 1-6, we will have a clear idea of what features ought to be included. The next step, Item 7, addresses how those features should work. I like focus groups for this detailed phase, but if budget doesn’t permit, we can cover this with informal round tables or virtual focus groups. I like qualitative techniques for this because it is difficult to get into enough detail with quantitative techniques.
When we wrap up Item 7, we have nailed all the requirements and are ready to develop a Functional Specification for Version 1 of the new product. This should represent a minimum set of features that service the mainstream workflows of end users in the primary persona. The product team can now execute on the new product.
The learnings from 1-7 will also feed Item 8 – roadmapping. We will have a ready list of features that deliver secondary benefits, and we can tentatively bracket those into future releases.
This process can sound like a drag. It is not. It is simply a practical way to frame the challenge of understanding customer problems. The whole thing can take anywhere from a few days to a few months, depending on how much the product team already knows about the market and the product, and the scope for the new product development effort. A little thinking goes a long way here – much better to invest time up front than start development without a clear understanding of what one’s doing and why.
Tags: Customer research, personas, qualitative research, quantitative research
This is the first post in my Customer Research series.
I will start with a discussion on why I do qualitative research. Some companies place a disproportionate amount of trust upon quantitative, statistically significant results presented in analyst reports. Anything involving a handful of research subjects is viewed with deep suspicion. Insights that their own product teams bring back from the field are dismissed as anecdotal, and the knowledge is not used for decision making.
In my opinion, this suspicion is dangerously misplaced. Numbers are sterile. They are necessary; any moderately skilled product person would stay on top of market sizing reports and industry trends by following analyst reports. However, that provides only a sliver of the knowledge needed to design the right products.
Qualitative techniques allow us to probe deep into personas, needs and wants, use cases, habits and practices and so forth with a small sample size. We step into the customers’ shoes and get a taste of their motivations, problems they encounter, and what they may be trying to achieve with a current or future product, in their environment with a workflow that works for them. This, coupled with quantitative research, is what drives great product development. It is incredibly time consuming, but if you plan and execute it properly, every minute is worth the work.
Please join the conversation if you are a product person – I would love to hear what you think.
Tags: beta studies, ethnographic techniques, personas, Product Development, product discovery, Product Management, research, usability research
I am doing a bunch of product discovery and product usage research at work. This is what I love most: getting out of the office and talking to customers. I decided to start a series on this topic since it is so much fun and so strategic.
For this series, I am using the word “customer” broadly – to include prospects as well as existing customers who have already purchased a product, and to include both buyers and end users. This is because there is no common definition of the word “customer” and this series addresses all of the above constituencies. I will clarify where necessary in specific posts.
Here is a tentative menu of posts. It’s a work in progress. Any suggestions would be greatly appreciated!
- Why do qualitative research?
- From personas to functional specifications
- Asking the right questions of the right people
- Common ethnographic techniques
- Using detailed interviews to build personas
- Focus groups vs. roundtable discussions
- Usability research in the lab
- Beta Programs
- On-line surveys
- Customer Advisory Boards
Tags: leadership, Small business, startup
Happy New Year! Here are my New Year’s resolutions. What do yours look like?
- Live more in the moment and enjoy what we have.
- Celebrate small victories.
- Spend more time listening and less time talking.
- Pause longer before reacting.
- Socialize and help facilitate socialization at work while maintaining productivity.
- Make professional development plans for all team members.
- Make good enough decisions based on good enough data, then iterate.
- Make it easy for everyone to keep the main thing the main thing.
I fly to Asia on a regular basis. New Year’s Eve found me in the airport, getting ready to visit Asia once again.
United was my second last choice (my absolute last choice is Northwest, which my Japanese friends lovingly nicknamed “Northworst”). I usually fly Cathay or Continental but the tickets were so expensive that I decided to give United another try. Big mistake!
I never was smiled at so much by United folks (usually they are quite burly). But I also never had such an unsatisfying flying experience before. One ticketing agent kept confusing whether I wanted a window or an aisle seat. Another took my passport and itinerary, and then spent the next 5 minutes chatting with a Massport agent. Since there was not enough room in the overhead compartments, a United employee whisked my bag away from me and stowed it somewhere without stopping to tell me where it was. The last straw came when a flight attendant opened the bin above me and dropped a computer bag on my head. (OUCH!)
Maybe this is why their tickets are cheaper than those from other airlines. You get what you pay for. Next time, Cathay!
*** January 5 update: ***
Today I discovered the United agent at Logan had messed up my return flight – I requested to move to a later connecting flight and she did (after 20-25m). Apparently (a) she forgot to cancel the 1pm, so I was double booked (b) she neglected to tell me there was a $175 fee to move to the 4pm. The United office in Hong Kong said I must call my travel agent and they can do nothing. I ended up Skyping out to United in the US to switch back to the 1pm return flight and negate the whole thing.
Am I the only one or is this obscenely absurd?