Customer surveys can be powerful tools when used in the right context, and when designed rigorously from the outset. They offer businesses a window into their customers’ minds, creating opportunities for improved products and services, and uncovering new business opportunities.
Likewise, when applied in inappropriate situations or put together poorly, surveys can be weak data collectors and potentially misleading.
As I’ve discussed, there are three problem areas where customer surveys are limited for gathering meaningful data, the first being self-reporting of reasons for past behaviour.
Here are the other two areas, and how to handle them:
Predicting future behaviour
Surveys are normally quite bad at predicting future behaviour as well. That has more to do with survey methodology and design than with the customer taking the survey.
If we think about pricing, for example, asking a customer to select one price from three available options for a new offering will normally result in the survey taker choosing the lowest price.
A better survey methodology for dealing with pricing is called conjoint analysis, which takes the survey taker through a series of trade-off decisions, where prices are tied to other elements of value, such as product features or quality levels. Only in context can customers make intelligent decisions about money on a survey.
Another example involves asking customers about potential new products, for instance, “Would you like to be able to buy a mobile phone from a vending machine?”
Again, out of context, it is very difficult for a customer to know how to answer that question. Instead, a much more descriptive narrative followed by a question is often required, where the conceived use is laid out.
In this instance, if the conceived use is transit hubs and phones for travellers, a better narrative followed by question would be: “Imagine you are travelling throughout Europe and you’ve just arrived in Barcelona late at night, without a mobile phone functional in Spain. Would you like to be able to buy a mobile phone from a vending machine in a public place?”
When trying to model future behaviour, surveys can be employed, but time and thought must be given to providing maximum context (without causing bias to the results) and potentially employing advanced analytics to get to better answers.
Determining why and how customers do what they do
Surveys are good at four of the five W’s: who, what, when and where.
These questions are typically fact-based, quantifiable or at least mutually exclusive, and, to a large degree, free from judgment on the part of the survey taker.
Why and how are the opposite: They involve opinion and are hard to put numbers around. It is difficult to obtain solid how and why data via a survey.
The limitation here is a practical one. It is very tricky, even if some qualitative research has been done in advance to inform the survey design, to adequately list reasons and answer options to how and why questions.
For instance: Why did you purchase your current motorcycle? Answer options: a) low price, b) great ride, c) high quality components, d) brand, e) other.
The real answer may be a combination of answers or parts of answers that will be listed, or lie outside of the listed options.
Leaving questions open-ended is preferable but just moves the issue of interpretation later in the process. It is then up to the analysis team to spot trends and put these answers into groupings, which is a complex, inexact science.
A more preferable means of tackling how and why questions in surveys is to instead ask many indirect, fact-based questions, and then correlate the answers to infer the answer to the problem.
For example, if customers are asked about the importance of a number of product attributes, and then asked about their satisfaction with those same attributes, it may be possible to determine which attributes drove certain decisions (e.g. high importance and low satisfaction hints at why a customer may not have chosen to repeat with a certain brand).
Depending on how the questions are set up and how the math is done, this can be known as “derived importance.” Derived importance is controversial as a methodology, as some research experts claim it is too big a leap from correlation to causation. However, it tends to be more accurate than asking customers how and why questions directly.
A different approach is to go off-survey for answers to why and how questions. Focus groups and/or one-on-one customer interviews tend to be better venues to get at how and why answers, since it is possible to ask follow-up questions and go on “fishing trips” with the customer in this setting.
Special to The Globe and Mail
Mark Healy, P.Eng, MBA, is a partner at Satov Consultants – a management consultancy with practice areas in corporate strategy, customer strategy and operations strategy. Mark’s focus areas inside the customer strategy practice include consumer insights, customer experience, innovation and go-to-market strategy. He is a regular speaker and media contributor on topics ranging from marketing to strategy, in telecom, retail and other sectors. Mark is known as much for his penchant for loud socks and a healthy NFL football obsession as he is for his commitment to Ivey and recent Ivey grads. He currently serves as chair of the Ivey Alumni Association board of directors. Mark lives with his wife Charlotte and their bulldog McDuff in Toronto.
Join The Globe’s Small Business LinkedIn group to network with other entrepreneurs and to discuss topical issues: http://linkd.in/jWWdzT