Usability research in the lab

[tweetmeme source=”chenelaine” only_single=”false” service=””]

This is the seventh post in my Customer Research series.

Usability research for consumer electronics products can be very costly.  There are companies that specialize in doing it the right way, with high end audio/video equipment and multi-stream video editing and compositing integrated into the program.  The deliverable is typically an incredibly insightful presentation with snippets of video that tell a compelling story all by themselves.

Since I work with startups and small businesses, I have never had the luxury of doing it “right”.   My theory is that some research trumps no research.  So I butcher best practices until they become unrecognizable but affordable (deepest apologies to Scott Weiss who taught me how to do it the right way!)

I usually start with a research protocol that clearly states the questions we want to answer, provides a guideline for recruitment and lists the props required for the session, which includes a mini-DVD camcorder and a tripod.   Then I design the session, which is videotaped throughout.  I try to stay inside of 1h if at all possible.

Let’s say I am comparing the usability of two smart phones for working moms aged 35-45, with at least one child under the age of 12 living in their house.  The session could look something like this:

  • Introductions and orientation – explain purpose of research to subject and let them know what to expect (5 min)
  • Execute any paperwork, such as an NDA, a photo and video release forms, and a profiling questionnaire (5 min)
  • Ask the subject to familiarize themselves with Device 1. Product manuals are provided to the subject. (5 min)
  • Repeat with Device 2. (5 min)
  • Ask the subject to execute a scripted task list for Device 1.  Tasks tend to be fairly specific – for instance, I could ask them to make a call, send and receive a text message, check traffic, take a picture, upload a picture to a computer, take a video, etc.  Ask them to verbalize what they are doing as they try things out (but do not offer hints or commentary – we are there to watch and learn, not to talk.)  (10 min)
  • Repeat with Device 2. (10 min)
  • Debrief – loosely guided interview to ask subjects to rate the usability of each task on a scale of 1 to 5, as well as answer some open ended questions about their general impressions and perceptions (20 minutes)
  • Present the incentive check (typically $50-100, depending on the nature of the study).

This format is great at providing a sanity check for the out-of-box experience for consumer electronics devices.  Can the end user figure out how to set up a new device and get it working without groaning and gnashing of teeth?  Lots of times they can’t.  I’ve learned so much about what’s wrong with the current packaging design and documentation from watching subjects struggle through product setup.   It is very hard to keep quiet and not offer suggestions along the way… but the learnings are priceless.

As with all other kinds of research, I am aggressive about inviting engineering team members to be observers in these sessions.  This is the best way to help them understand who they are designing for and why certain feature enhancements are necessary to ensure an awesome user experience.

Add to DeliciousAdd to FaceBookAdd to StumbleUponAdd to Twitter


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Twitter Updates


%d bloggers like this: