Archive for the 'Customer research' Category

Book Review: “Disciplined Entrepreneurship” by Bill Aulet of MIT

My friend and colleague Bill Aulet (currently Managing Director of the Martin Trust Center for MIT Entrepreneurship and Senior Lecturer at the MIT Sloan School of Management) has just launched his new book, “Disciplined Entrepreneurship – 24 steps to a successful startup“.


There are a million books out there teaching various aspects of entrepreneurship, many of them I consider required reading for any aspiring entrepreneurs in today’s startup ecosystem. But I agree with Bill when he says that all these books and materials teach only a part of the whole. First timers need a comprehensive introduction to the whole tamale. Which Bill does fantastically in a nice dose of MIT style firehose.

Bill hammers home the importance of testing your assumptions in the field. Other books do this too, but Bill makes it practical.  The single most fatal mistake an entrepreneur can make is to get so attached to their starting idea that they push forward despite warning signs along the way. A rigorous approach to testing assumptions is a powerful weapon against getting caught up in the focus group of one.

What I particularly like about this book is the approachable way Bill demystified the sales process, explained sales models, and outlined how to quantify and track key metrics like LTV (Life Time Value for a customer) and COCA (Cost of Customer Acquisition) beyond the blunt instrument of a financial statement. The chapters on sales and marketing are particularly valuable to technical founders who haven’t personally seen marketing and sales in action.

Another wonderful thing about the book? It doesn’t discuss product roadmap until Step 24 of 24 – which is exactly apropos. First, you establish your right to exist. Then, you can worry about product roadmap. A product no one buys is not a business. Fixating on future products before the value proposition has been proved on the first offeringis a fool’s errand.

Here are some quotable (tweetable?) quotes that I particularly like:

  • “The single necessary and sufficient condition for a business is a paying customer.” (On why having a product no one buys doesn’t make it a business.)
  • “Focus is the most important skill for an entrepreneur.” (On not boiling the ocean)
  • “If there is already a market research report that contains all the information you need, it is probably too late for your venture.” (On why one must do primary research instead of relying on Google search to learn about the market and customers)
  • “Get started doing, rather than getting stuck in analysis paralysis.” (On not agonizing over whether you have picked the right beachhead. You have incomplete data – just make a call and get going.)
  • “(Entrepreneurs) overestimate the enthusiasm their customers has on their product.” (Touché.)
  • “If the only feedback you get is ‘everything is okay’, then it is likely the customer doesn’t care much about your product and its value to them.” (On anticipating and embracing negative feedback)
  • “You cannot will a market to exist any more than you can change the laws of thermodynamics.” (On the need to be willig to change fundamental assumptions, including the choice of a target market, based on new data)
  • “Free is not a business model.” (Enough said.)
  • “Costs shouldn’t be a factor in deciding price.” (On how cost based pricing almost always leaves money on the table.)
  • “It is always easier to drop the price than to raise the price.” (Duh!)
  • “A sound ratio for the LTV to COCA is 3:1.” (Via David Skok – a good guideline for unit dynamics once a startup has reached product market fit.)
  • “The COCA will almost always start at a very high point because you need to first create the market.” (Something a lot of first timers don’t realize, causing panic, when it is part of the process to develop the business model.)

I very much enjoyed reading this book and the many examples.  If you are a first time entrepreneur who wants to learn what questions to ask, definitely add this book to your collection. It is a dense read, but it will arm you with the vocabulary and concepts to probe deeper in each area. Highly recommended.


When an idea is 10+ years ahead of its time

[tweetmeme source=”chenelaine” only_single=”false” service=””]
CES is happening this week. Since I’m a tech geek, I’ve been avidly following news and blog posts about all the gadgets and technology trends that are being announced.

So far the thing that interests me the most is the connected cars phenomenon. Mercedes-Benz, Audi and Ford each have their own proprietary platform that connects to the internet and allows you to get information from the internet and/or use your mobile phone to check the status of your car. This seems to be an idea whose time has finally come. Here is a video showing the Mercedes-Benz system. (As a former Audi owner I would have loved to show the Audi system instead… but this video is more digestible.)

This is especially interesting to me because some 15 years ago, while I was with a product design consultancy, I was part of a team that worked on an “infotainment car” concept with a cutting edge automobile company that shall remain nameless. We did an ethnography study where we shadowed research subjects for an 8-hour day as they drove around, going about their normal business with us and our videotaping equipment in tow.

We crunched the data, and came up with what we thought people would want to do in the car: get location based information such as nearby restaurants, get turn-by-turn navigation help, get entertainment such as music and video in the car, and get help and support if they get into trouble. We understood that the user’s eyes have to be on the windshield and we thought of wacky ideas like a heads-up display (HUD) superimposed on the windshield, so that critical information may be presented to the driver without requiring them to take their eyes off the road. We called this the “infotainment car” concept.

Considering this was mid 1997, US cellular networks were in the dark ages (GSM/GPRS was not even approved as a standard), and the most advanced connected car technology on the market at the time was General Motors’ OnStar system (equipped with a GPS, an analog cellular uplink and people answering calls), the infotainment car concept was truly a glimpse into the future.

We eventually visited the automotive company’s advanced research lab and saw such a concept car with all the requisite technology. This concept car would have worked from a technology standpoint. Its only problem was that the technology was very expensive and far from mature, and the content and infrastructure was sparse, and in some cases, non-existent. The ideas were wonderful and in hindsight, more than prescient, but the content and technology limitations made it impossible to realize the full richness of the user experience. The concept car stayed in the lab for another 10+ years.

Fast forward to today, and look how far the technology has come. The cellular uplink is now smoking fast – witness the 4G LTE radio integrated into the Audi. This makes it possible to have a really great data download and media streaming experience. Location based information is accurate and plentiful. Many people (myself included) keep large amounts of personal data in the cloud, making it ever more possible to have an excellently consistent connected experience anytime, anywhere. Advanced display technologies are starting to become a reality. Audi is even talking about a heads-up display.

Here is proof that an idea alone isn’t enough to make a successful product or business. Execution alone isn’t, either (the automotive company knew how to make such a car, albeit at a crazy price point). The right external conditions are the third requirement.

I have another case study that supports this observation. Another company I worked with that shall also remain nameless had thought of the exact same core idea as the MakerBot Replicator. Here is a video from the company explaining this incredible 3D desktop printer.

That idea came up well over 10 years ago and remained unactionable until recently, when relatively more cost effective 3D printing technology, better options for the substrate, as well as 3D digital content creation technology caught up with each other.

As a product person, one must stay on top of technology trends and be alert and aware when the conditions arrive that makes a brilliant but previously impractical idea come into its own. Being in the right place and in the right time is a pre-requisite to success.

Product Planning Series: Requirements

[tweetmeme source=”chenelaine” only_single=”false” service=””]

This is the sixth post in my Product Planning Series.

The mere word “requirements” can make a lot of startup people wince. It conjures up the bad old days where folks spend months developing an MRD, PRD and a Functional Specification. It brings up images of Stage-Gate and classic Waterfall processes.

There are situations where big long planning documents make sense.   For example, if you are developing a medical device that needs to go through the 5(10)k process, you really have very little choice.  You have to go by the medical devices handbook which pretty much stipulates that you need to write all those documents and place them under a document control process.

However, in a startup situation, where you really don’t know if your problem, solution, and business model will jive with customers and users, overinvesting in planning documents just leads to lost time and productivity.  It also sends the wrong message to the team: instead of being open minded and work closely with customers to define the product and the business model, you are making up too much of it in the office. By the time the big MRD is written the world has already changed and the requirements could be obsolete.

I am against over-investing in requirements documents. However I am even more against not writing anything down and relying on tribal knowledge to implement a solution. That works if there are, like, 2 people in a startup. When you have more than 3 or 4 people working on the same thing, communications becomes very important. Working in a fast paced startup where new data comes in constantly is not an excuse to punt on basic common sense.  Good team communications practice begats good project execution, which will increase the probability that your product will do what you want it to do in the marketplace.  By the way, this goes for hardware AND software.  Even an agile process needs a holistic view of the end goal.

Here are some requirements do’s and don’ts in a startup setting.


  • Write SOMETHING down
  • Get buy-in from ALL STAKEHOLDERS.
  • Be practical and specific. Leave it loose at your own risk.
  • Build a top level project plan that identifies key tasks, milestones and interdependencies
  • Develop a lightweight 2 year roadmap
  • Start building anything without buy-in
  • Write a 100 Page MRD
  • Overspecify details on each feature in classic Waterfall fashion
  • Build a 5 year product roadmap with a great deal of detail
At the end of the day, for a development team to be productive, they really don’t need big long documents to describe everything. They just need a few slides on a few topics.  Here are the topics that I find useful to write down for the team. A lot of this stuff should be available for free, from the business case and who/what/why/when discussions.  Some of it is technical – like a lightweight description of the MVP.
  • Clear description of the market problem that is being solved
  • “Elevator pitch” of the solution (preferably with images)
  • Description / analysis of first target segment
  • Buyer and User Personas
  • Detailed storyboards on top 1-3 typical workflows
  • Specific examples for details encountered in each workflow
  • Considerations for human factors / human cognition
  • Top level design directions to be followed by product design team
  • Lightweight functional description of minimum viable product (MVP)
  • Quick and dirty 2 year product roadmap
  • Any external business drivers (e.g. trade shows, funding runway, etc.)
The most important thing is to do the thinking and research that will support the materials presented in these types of documents. The second most important thing is not to overthink things when you write them down.  Use the minimum possible number of pages to convey the information – don’t overinvest.  That way you will not feel too badly if customer development tells you something new and you have to trash some of these documents and write them over.

On brand loyalty and the iPad 2

iPad 2 versus Samsung Galaxy Tab 10.1
[tweetmeme source=”chenelaine” only_single=”false” service=””]

I have mixed feelings about mobile products from Apple.  On the one hand, they are gorgeous.  The hardware is well designed and manufactured, the software interface is fabulous, and the user experience simply delights.  On the other hand, they are made by Apple, and I have a serious problem with their business practices.  

Apple’s closed ecosystem completely turns me off.   But it’s the experience of doing business with Apple on behalf of a previous employer that has permanently soured my ability to truly enjoy Apple products.  To this day I cannot look at an iPhone or an iPad and not get flashbacks.

This is why I carry an AT&T Samsung Captivate running Android 2.2 (Froyo), and conduct all my business using Google apps on my phone.  Samsung and Google are two brands that I dig.  I find Samsung to be almost as good as Apple in hardware product design.  Their mobile business unit is highly innovative and the Galaxy and Galaxy II lines are fantastic.  The cosmetics, fit and finish of their phones are impeccable.  I like their oversaturated displays – even my friends who are die-hard Apple fans have had to concede that the 4″ AMOLED display on my phone is more vibrant than their iPhone 4 display. And It’s Not Apple.  

As for Google, I can’t exactly remember life before Google Apps.  Even my grocery shopping lists are kept as a Google Doc.  And That’s Not Apple, either.

This personal hangup explains why I did not get an iPad2 the minute I decided I needed a tablet device.  Instead I spent weeks researching available choices: Acer IconiaAsus TransformerMotorola Xoom? … etc.

I ended up convincing myself that the Samsung Galaxy Tablet 10.1″ would be the right choice for me.  This thing looks breathtakingly on paper. It’s the same size and weight as the iPad2 (thinner and lighter by a hair – not a meaningful differentiator), but it’s not made by Apple.  

So I eagerly waited for the device to come available at my neighborhood Best Buy on Friday 6-17-2011.  I ran to the store on Saturday in order to play with one, benchmark my experience against the iPad2, then buy the Samsung tab.  I thought it was a shoo-in.

Instead, I found that the device was stunning in every hardware detail… but (GASP AND ALAS) I liked the iOS experience on the iPad2 far better than the Android Honeycomb 3.1 experience on the Samsung Galaxy Tab.  Honeycomb is a fine OS, but it doesn’t hold up a candle to the usability of iOS 4.  And we’ve all read about the wonderful advances in iOS5, coming soon to an Apple device near you.  So I went home depressed… and ordered an iPad2 on line.

If ever there is a cautionary tale about the fickle nature of brand loyalty, this is it. I really, really wanted to get an Android tablet, made by Samsung.  But in the end, the better product won.

Moral of the story: invest in the user experience.  Make it delight the end users.  It will pay off in the end.

Product Planning Series: From use cases to storyboards

[tweetmeme source=”chenelaine” only_single=”false” service=””]
This is the fifth post in my Product Planning Series.

My approach to product development revolves around user-centered design. The basic tenet of this philosophy is that the product team must be equipped with a thorough understanding of the end user’s needs, wants, expectations and limitations in order to create an excellent product solution to solve the user’s problems.

This understanding begins with user personas at a high level and becomes fleshed out via use cases and user stories.  The UX design team can then ideate on a solution to the problems the user is trying to solve and create storyboards to imagine how the solution may be implemented in the context of the product.

The words “use case”, “user story” and “storyboard” can mean different things to different people.  This is what I mean when I use these words to describe the tools and artifacts I use in the product design process.

  • Use case: A high level thought experiment of a workflow from a user’s perspective.
  • User story: A brief description of a part of a use case or storyboard that succinctly defines a task the user has to complete, from the user’s perspective, with no assumptions placed on design or implementation. Used to describe functionality that will go into a backlog to be prioritized and managed by a product owner (in classic Agile methodology)  It is usually much more granular than a use case and describes a snippet of what the user needs to do to complete a workflow.
  • Storyboard: An output of the design process that illustrates the experience of the user in a journey to complete a workflow using the product. In my experience, this is the fastest and most effective way to turn a high level UX idea into something concrete that the product management team can use to test with customers and the product development team can use to plan their work.
I find that these three things are fairly universal in their applicability to all kinds of products, hardware and software alike.   In cases where the use cases are complex and the solution is non-obvious, it is vastly faster and cheaper to iterate a design idea at the storyboard level than to code it up and then review the actual working code output.
With the right talent on the task, one can literally come up with 5 or 6 storyboard iterations in a single day without investing in any engineering development.  The impact on software products is substantial – it can take weeks to program just one of those iterations, so storyboards saves time and money in a tangible manner.  The impact on hardware products is game changing – a single iteration for a hardware implementation could take months or longer.  Storyboards allow the product team to iterate, test and learn, so that we can come up with a better end result in a shorter period of time with the least possible investment in engineering development.
Once the product design is vetted at the storyboard level, it becomes much easier to fill the rest of the design gap with a detailed design solution, and development will then be able to implement the design efficiently and effectively.

Product planning series: who, what, why, how

[tweetmeme source=”chenelaine” only_single=”false” service=””]
This is the second post of my Product Planning series (which uses a new web app as an example to illustrate how you go about planning the release of a new product or service.)

In my last post I outlined some of the questions that you should ask to establish a business case for your new product or service. Once that’s done, it’s time to define who you are building the product for and the problems they face, what your proposed solution looks like, why you think it is better than anything else out there, and how your solution will work to solve these customer problems.

The first question some people ask is: why segment and target? If one does a wonderful job with product development, shouldn’t the product sell to anyone who might find a use for it? My answer is this: it is not possible to develop a wonderful product without knowing who you are designing it for.

Let’s say our web based B2C consumer SaaS offering is an educational site for children that focuses on advancing the musical education of kids who are learning an instrument.

Right away we can see that the buyer (a parent, grandparent or teacher) is different from the user (a student). You will need to build a buyer persona and a user persona. The buyer (parent) wants the child to learn. The user (child) just wants to have fun. To be successful the product must both educate, to meet the needs of the buyer, and entertain, to satisfy the wants and expectations of the user and to ensure stickiness and compliance in product usage.

If the word “persona” is new to you, Pragmatic Marketing has an excellent article explaining the persona concept, and Scott Sehlhorst has a great article on this topic. You can also consult my post on using ethnographic techniques to develop personas.

The purpose of developing the persona is to use the target buyer and user to help decide what to build and how to present it. What age group are we targeting – do you need to cater to the pre-reading crowd? How high do we go in the age bracket? These affect the user interface and use cases because clearly a 15 year old violinist playing first violin in a youth orchestra has vastly different needs and wants than a 5 year old violin novice working on basic bowing techniques.

Persona development goes far beyond demographic information and goes deep into situational scenarios. What is it like living in the households of these music students? What is their daily schedule? How long do they spend on music practice each night, and for how many nights a week? When do they practice music? How involved is the parent? Are they distracted by their siblings? What parts are hard to learn? What are the objectives of the child and what problems does he or she face? What are the objectives of the parents and how do they measure their child’s success?

A good persona provides demographic and psychographic information, as well as additional information about the attitudes and motivations of the persona in the area of product use. It is well worth the initial investment to develop a good set of personas, then clearly delineate the problems they face. It will help make development more efficient down the line and help you develop a great user experience that is tailor made to your target end users.

Once the buyer and user personas are defined, and their problems are well understood, it’s time to figure out what the solution to these problems might look like. For example, it is very hard to motivate a young child to learn all those pesky Italian musical terms. Perhaps an on line memory game might help them remember those terms. Another example is that the child may need to work on posture. Perhaps a video feature that allows the parent to take a video with their cell phone, then upload to a server and share it privately with the teacher might help. A third example is that the child has forgotten the correct technique 3 days after the teacher demonstrated it. A video snippet of the lesson might just do the trick. And having this all in one place encourages usage and compliance.

As a product team comes up with solutions, there are invariably way more ideas than what would fit in a desired timeline. Here is where the persona will be useful: the product team can use the persona to help them imagine what is of the most use to those personas, and develop a release roadmap where features that deliver the most benefits and value are released first, followed by other features that either broaden the offering or offer secondary benefits.

Many wise product and startup people, including the folks at Y-Combinator, have commented on the need to focus on the customer, not the competition. This is very true. However that doesn’t mean you get to punt on doing a thorough competitive analysis to understand exactly what is out there in the market. This helps you in several ways:

  • You can check to see if your solution really solves an unmet need. If to your chagrin you find something that solves the problem quite well, it’s better to know sooner than later so you can pivot to solve a different problem that REALLY represents unmet needs.
  • You can learn from the successes and failure of other people who were in the market before you.

With a good understanding of the competition you are well poised to articulate your competitive advantage which is necessary for developing an actionable and meaningful positioning statement.

Interestingly, many technology startups start with the “how” as the basic premise for starting a company. Someone comes up with a brave new technology, develops a prototype for it, gets it to work, falls in love with it, then sets off starting a company with a field-of-dreams business plan.

That works sometimes if the technology is really world shattering. In consumer SaaS, the technology itself is sometimes commodity software. So for those types of products, the “how” comes only after having figured out the who, what and why. The “how” is concerned with how to actually implement the solution. For this example, you would pick the technology platform to develop your new site on – fielding considerations such as open source versus proprietary platforms (my vote: Open source), database of choice (I like MySql), programming platform (for complex web apps: Java EE back end with Javascript or HTML5 front end; for quick hacks, I like Ruby on Rails). You will need to make decisions early on about hosting and server care and feeding too: certain platforms bring a hefty server side license fee and you will need to account for it in your projected server fees and expenses.

Once the platform is picked, you can now work on defining and designing the product in detail, staffing up for success, and executing the plan to bring your offering to market in the desired timeframe.

Price testing: monadic purchase intent test

[tweetmeme source=”chenelaine” only_single=”false” service=””

This is the tenth post in my Customer Research series.

Today I read a fantastic post on the Vovici Blog by Jeffrey Henning on monadic price testing.  You can read it here.   In fact this is such a fantastic blog that if you are interested in customer research at all, you should subscribe to the feed – I haven’t found any post that’s not insightful and educational on this blog yet.

The post on monadic testing was particularly apropos for today since we had some new team members come on board, and we reviewed the methodology and outcome of a purchase intent at price study we did early last year that used this methodology. In a nutshell, here’s what you do:

  • Come up with a screening questionnaire that will only let in the respondents you care about (e.g. you will want to screen for people that fit your primary buyer persona.)
  • Come up with a “product concept” that describes your product or service’s key value proposition. This tends to be a two page text and image based presentation in an on-line survey.
  • Pick the price points you want to test.
  • Send out the survey to several groups of respondents, each of which will see the exact same product concept, at one of the prices you want to test.  Respondents are asked to rate their likelihood to purchase on a 5 point scale.
  • Keep collecting completed surveys until you have at least 100 samples per price point (Best practice suggests collecting 100-400 samples per cell).
  • Now tally the results and see if the top-two-box scores (percentage of people who respond with “very likely” or “completely likely” purchase intents) show any kind of a trend as you vary the price.

A lot of times people assume adoption will rise as price drops. In reality that’s true only if the product or service is not valued by the target respondents or if there is an obvious comparable product that has a low price.  If a monadic study like this shows no obvious trend when you vary the price for the exact same product concept, that is a clear cue to product management to hold the price and not cave in to channel pressure to drop the price.

The reality is that retail and other channels will always pressure a business to drop the price – but that’s not always going to result in an uplift in customer acquisition.  If the customer has substantial pricing elasticity, then dropping the price only serves to reduce your margin, and doesn’t necessarily buy you additional market share.  Something to keep in mind when thinking about how to price your products and services.

Want a killer UX? Wireframe and iterate

[tweetmeme source=”chenelaine” only_single=”false” service=””]
This post is about how to iterate on UI and deliver a stunning user experience to end users on a shoestring budget. The secret is to do it the old fashioned way: wireframe it first, then make compositions, then test the storyboards with users, then code it up.

I’ve been part of several development projects where my project team was forced to cut corners on up front design.  Sometimes the product management and design teams were busy doing other stuff.  Other times (GASP!) the company didn’t actually have product managers or designers.

Typically the feature would be communicated in the oral tradition.  If we are lucky we sometimes get napkin sketches.  Engineering is then requested to code it up so we can “see what it looks like”.  The requirements are generally very sparse, marginally communicated, and the user workflows are not thought through.

This of course results in something with a UX that is at best rough around the corners and at worse inexplicable.  Changes are requested, new code is written, and this cycle repeats itself until the project overrun is intolerable and the feature is released to the wild with multiple compromises.   The project ends up taking 2 or 3 times as long as it should.  We get lots of 1 star reviews.  Everybody is cross with everybody else with how things turned out.

Instead of rushing to code stuff up, if the PM and designer take the time to start with user storyboards, flow charts and wireframes, iterate until mainstream use cases are solved, then move to the compositions depicting the graphical layout of every screen in the feature, and then validate with end users, the exact same project would take half as long in elapsed time and will work 10 times better for the end user.

This is because the PM and interactive designer have the best access to actual end users, and are the most qualified people in the company to dictate how the feature works.  Also, tools like Microsoft Visio makes it possible to do 20 iterations on a wireframe in a day, whereas it is simply not possible to iterate that fast in code.

In a startup environment, we sometimes measure success by the wrong metrics: we equate speed to market to success.  A better metric would be speed to market with a great user experience and satisfied customers.    It’s much better to invest up front in a good design – you will save money and time in the final analysis.

How to conduct a competitive RFP

[tweetmeme source=”chenelaine” only_single=”false” service=””]

In recent months I have had three opportunities to conduct a competitive RFP on three different initiatives. I have learned so much about this process. Here are some key takeaways:

  • If you are looking for someone or something new, ALWAYS comparison shop. It takes twice as long.  But you will think through the selection criteria much more clearly, have a much deeper understanding of each candidate’s strengths and weaknesses, have much better leverage on price, and will almost always save money in the long run.
  • Make sure you write a thorough RFP and take special care in assembling the supporting documentation.  A proposal or quote is only as good as the RFP package.  Clearly outline what you are asking someone to quote, break it down into phases, make it very clear how you want the response to be structured, and include an RFP questionnaire to ensure the response covers key areas of interest.
  • Where possible, do a budgetary quotation on something you know the price of. For example, if you are looking for someone to develop a new iPhone app, you could have them look at an existing iPhone app and ask them how much it would take for them to create that app.
  • Give vendors enough time to put forward a thoughtful response.  A week or two is appropriate for most projects.
  • After you receive each response, follow up with a phone call to go over each line item, ask the candidate partner to clarify any ambiguities, and clearly express your issues and concerns.
  • Respond to their response in a timely and respectful manner.  They have worked hard to provide you with a quality response, and they deserve to be treated professionally even if you have decided against working with them.

The dreaded “dark side”

[tweetmeme source=”chenelaine” only_single=”false” service=””]

I am currently working on a product and service combination where there is a hardware component and an online software component.  Predictably, our customers are split between those who go on line and those who don’t.  Also predictably, we know vastly more about our on-line customers than our off-line customers.

This really came to a head when we were doing customer satisfaction surveys.  Almost all respondants are on-line users.  But what about the “dark side” – our off line users?  Are they happily off line, or did our product become a paperweight?

Of course, there is only one way to find out: interview them by phone.  These folks have proved to us that they don’t go on-line to use our software tools, or respond to our emails.   They most definitely won’t post anything on a forum or tweet us their thoughts.  We would have to revert to old fashioned phone calls.

The issue is the time commitment: we can blast off an online survey to on-line users any time we want.  We can always squeeze one in no matter how many other initiatives we are chasing.  But to get 20 quality phone interviews from the dark side, we are probably going to have to actually call 200-400 people, maybe even more.  The time commitment is so immense that it nearly always get postponed when the subject comes up.

Some time soon, we’ll bite the bullet and call our “dark side” customers and find out what they love or hate about our overall user experience.  It’s no different from how anyone else did primary market research 20 or 30 years ago.  It’s just interesting to observe how our expectations have changed with these awesome on-line tools. We are so spoiled 🙂

Twitter Updates


%d bloggers like this: