We conduct a lot of customer research. It is at the heart of most of our projects. And surveys are part of the research design for more than half of these studies.
I was reflecting last week with a colleague in media on what we had learned over the years about customers, and the conversation turned to the advantages and limitations of customer surveys.
Customer surveys can be powerful tools when used in the right context, and when they are designed rigorously from the outset. They allow businesses a window in their customers' minds, creating opportunities for improved products and services, and uncovering new opportunities.
Likewise, surveys can be weak data collectors and potentially misleading when applied in inappropriate situations or put together poorly.
There are three problem types where customer surveys are limited ways to gather meaningful data:
1. Self-reporting of reasons for past behaviour
2. Predicting future behaviour
3. Determining why and how customers do what they do
Careful survey design, or alternative and non-traditional research approaches, must be employed to tackle these problem areas.
Self-reporting of reasons for past behaviour
Surveys are solid tools for gathering facts, especially in the present (how many times per month do you shop at…?) or in general (which colour of car do you most prefer?). Behaviours – particularly past behaviours – are much more difficult to accurately measure via survey because people will later rationalize irrational decision making.
We did some work in the chocolate-milk space a couple of years ago and employed in-field observation and intercepts (convenience and grocery stores) for data gathering. We would ask consumers why they chose the carton or bottle they selected, a few seconds after the customer had pulled it from the cooler. A number of consumers would say something like “I picked the one with the latest freshness date,” and we'd counter with “no you didn't, you just reached in and grabbed the one closest to the front.”
Only after confrontation would consumers say “oh, you're right, I guess I liked the package” or “I was in a real rush.”
Consumers don't lie, but their brains do want to put order and process around decisions where the intent and decision drivers were different – so asking similar questions on surveys can lead to bad data and inaccurate interpretations. In this case, non-traditional qualitative research can yield better results.
Predicting future behaviour
Surveys are normally quite bad at predicting future behaviour as well. That has more to do with survey methodology and design than with the customer taking the survey.
If we think about pricing for example, asking a customer to select one price from three available options for a new offering will normally result in the survey taker choosing the lowest price. A better survey methodology for dealing with pricing is called conjoint analysis, which takes the survey taker through a series of trade-off decisions where prices are tied to other elements of value, such as product features or quality levels. Only in context can customers make intelligent decisions about money on a survey.
Another example involves asking customers about potential new products – “would you like to be able to buy a mobile phone from a vending machine?” Again, out of context it is very difficult for a customer to know how to answer that question. Instead, a much more descriptive narrative followed by a question is often required, where the conceived use is laid out. In this instance, if the conceived use is transit hubs and phones for travellers, a better narrative-followed-by-question would be: “Imagine you are travelling throughout Europe and you've just arrived in Barcelona, late at night, without a mobile phone functional in Spain. Would you like to be able to buy a mobile phone from a vending machine in a public place?”
