Article

The top 10 reasons you don't understand your customers

Our top 10 list of the reasons why customer surveys don't work.

By Fred Reichheld

First published on Μαΐου 01, 2006
min read


My favorite customer-satisfaction survey? The one from a high-end hotel overlooking New York City's Central Park, where the rooms average $700 a night. Thirty-five questions ask you to rank the valet, the greeting at reception, the look and feel of the lobby, the concierge, the restaurant, the room service, the bathroom, and everything else imaginable. Then the survey poses a truly absurd query: "To what degree do you agree that this is a luxury hotel?" The fact is, satisfaction surveys have become a long-running, bad joke. Far from informing companies of what their customers truly value, they are becoming a major reason that organizations misunderstand their customers entirely. In the best tradition of late-night comedy, here are the Top 10 reasons they don't work. 10. There are too many questions Like the hotel, companies pile on the questions. That raises the cost of the survey, discourages would-be respondents, and shrinks the sample size. It can be just as bad in business-to-business contexts. Is it any surprise that engineers get irritated by a 130-question e-mail survey from a leading technology company? One wag calculated that if 85,000 employees spent 26 minutes per survey, that would expend almost 37,000 labor hours, which he valued at $100 per hour. Result: The survey would cost nearly $4 million in engineering time. 9. The wrong customers respond As a manager, do you really want to hear only from people who have nothing better to do than answer lengthy surveys? Chances are your most profitable customers ignore them. Surveying the wrong customer is a particular problem in the business-to-business arena. For example, many employees may use business software, but it is usually a handful of executives who make purchase decisions. When software vendors carpet-bomb customer lists with surveys, busy executives are apt to ignore them or delegate them to junior clerks. Most respondents will be people whose priorities don't necessarily match those of the managers making the decisions. 8. The right employees never hear the problems If a customer does report a source of dissatisfaction, the information never reaches the people who can make it right for her. Instead, she becomes just one more data point in a marketing report. Indeed, how can frontline employees be expected to respond to the deluge of replies to all those questions? Even if they did have the time, they would probably be responding to the needs of the wrong customers. Customer feedback needs to come from customers with enough economic value to merit investing in solutions to their concerns. But customer anonymity is a hallowed principle of market research, so there isn't even the possibility of follow-up questions to probe root causes and possible solutions. 7. Many surveys are sales calls in disguise Marketers have learned that prospects are less prone to hang up when asked a few initial questions than when they get an outright cold call. Result: Customers learn to mistrust anything that sounds like a survey. In fact, many firms commission surveyors to call customers but have no intention of fixing customers' problems or improving their experience. For instance, several auto manufacturers discovered they could boost their J.D. Power and Associates customer satisfaction ranking by calling all customers two weeks after they purchased a new car to ask if everything was OK. The manufacturers hired a vendor that had no way to take action on the feedback, because the real purpose of the call was to prime customers for a subsequent call from J.D. Power surveyors. 6. Survey scores don't link to a company's performance If they did, Wall Street would be the biggest consumer of survey data. As it is, investors pretty much ignore customer-satisfaction reports. Bain research teams consistently find that the links between satisfaction-survey scores and customer behaviors that drive profitability or growth are tenuous at best. Detailed analysis of individual customers, for example, typically finds that 60%-80% of customer defectors score themselves as "satisfied" or "very satisfied" on surveys preceding their defection. Moreover, companies that achieve satisfaction scores of 80% or even 90% often experience no economic advantage from their apparent customer loyalty. 5. Plain-vanilla solutions don't fit companies' needs Survey-research firms often push cookie-cutter solutions. But does a bank need the same information as an auto manufacturer? Predictably, research firm customers end up with useless generic data. Recognizing this, Pat Flood, CEO of Atlanta-based HomeBanc, had HomeBanc develop its own process: the company's closing attorneys give each customer a brief survey right after closing the loan-at the point at which any glitches in the process are fresh in the customer's mind. Almost 70% of customers respond, and the results are linked to the bonuses and promotions of each employee who interacted with that customer. 4. Everyone gets to make up their own rules The new CEO of a global telecom company found that each country and business line in the company had its own satisfaction-survey vendors, its own methodology, and its own schedule for reporting results. If a division didn't like what it heard, the solution was simple: tinker with the methodology. (The CEO insisted on one set of basic rules for gathering and reporting information, but those rules were applied to make sense in the specific business settings.) Indeed, debates about survey methodologies run an amazing gamut. Consider the search for the best scale for measuring and reporting customer feedback. Some experts argue that a simple yes or no structure is best. Others advocate a 5-point scale where 1 means excellent, 3 represents neutral, and 5 means poor. Still others prefer to reverse that 5-point scale, making a score of 1 equivalent to poor, and so on. And some vendors don't provide a "neutral" box, forcing customers to choose between positive and negative. 3. Surveys confuse transactions with relationships Gauging the quality of a single transaction is easy. But evaluating a relationship has to include every detail of the customer's experience—awareness, purchase, servicing, billing, upgrades, and so on. Companies rarely assess all these touch points. One large bank's first attempt at building a reliable relationship metric simply tallied up all the transactional satisfaction scores. Not surprisingly, these didn't tell the bank's managers much. Only when they began to home in on the goal of creating enthusiastic, delighted customers did they learn what mattered most to their core customers and focus on delivering those things to them. 2. Surveys actually dissatisfy customers Who wants to be interrupted at dinner or badgered at work, just for a survey? A friend who bought a new car says he'd be more satisfied if the company just cut prices by the amount it spent on the multiple surveys he received. Most managers hire contractors who have little knowledge of customer needs or the sponsor's business. But it doesn't have to be this way. At Harley-Davidson, for example, customers are treated like family; they get phone calls only from recent Harley retirees (hired back part-time) who know the company and its products well and who are charged with listening closely to customers. Not coincidentally, these retirees generate deeper customer insights while also reinforcing the Harley culture and brand. At Southwest Airlines, President Colleen Barrett discourages large-scale surveys and instead insists that any employees who want feedback from a customer write a personal letter requesting that information and explaining what they intend to do with it. They must also write a thank-you to customers who respond, describing the actions that will be taken as a result of their feedback. Southwest also gains plenty of feedback from face-to-face interactions with customers. 1. Manipulation wrecks surveys' credibility Think of your last visit to a car dealership: you probably saw an enlarged satisfaction survey on the wall, with all the top boxes suggestively checked. Turns out this boosts dealerships' ratings. Dealers even get salespeople to pressure customers for top ratings, often at the cost of providing free floor mats or oil changes in return. Some savvy customers have learned to play these games, too: they negotiate a low price and then offer a high satisfaction score to get an additional $500 off. After spending millions of dollars on satisfaction surveys, the car manufacturers have ended up with a system that irritates customers, turns salespeople into cynics, and fails to provide reliable scores to link satisfaction ratings with dealer profits or growth. The truth is, meaningless measures of customer satisfaction don't help companies grow. What a business needs isn't satisfied customers but passionate advocates who sing the company's praises and who wouldn't dream of defecting to the competition. Gather regular, systematic, timely data about customer advocacy and make that information available to your operating managers right away. The good news is that you can gather this data with as little as one question—I call it the "ultimate question"—and that's "How likely are you to recommend this company to a friend or colleague?" Ask this question regularly and systematically, then score the results on a 0-10 scale. You'll find out how many of your customers are loyal promoters (9s and 10s), how many are just passively doing business with you (7s and 8s), and how many are detractors (everyone else). The key to growth? More promoters, fewer detractors. Keep a close watch on your Net Promoter Score (NPS), the percentage of promoters minus the percentage of detractors, and you'll start to understand how many and which customers constitute your secret sales force. Companies such as GE have adopted NPS because it opens a window into how customers think about their organization. Call the profitable customers who are happy, and understand what you are doing right so you can do more of it; call those who are unhappy, and you'll learn what it takes to improve your offering and sustain profits, which is where the hard work begins. In any case, you can forget those cumbersome satisfaction surveys entirely. Purchase the full text of this article from Harvard Business Online.

My favorite customer-satisfaction survey? The one from a high-end hotel overlooking New York City's Central Park, where the rooms average $700 a night. Thirty-five questions ask you to rank the valet, the greeting at reception, the look and feel of the lobby, the concierge, the restaurant, the room service, the bathroom, and everything else imaginable. Then the survey poses a truly absurd query: "To what degree do you agree that this is a luxury hotel?"

The fact is, satisfaction surveys have become a long-running, bad joke. Far from informing companies of what their customers truly value, they are becoming a major reason that organizations misunderstand their customers entirely. In the best tradition of late-night comedy, here are the Top 10 reasons they don't work.

10. There are too many questions

Like the hotel, companies pile on the questions. That raises the cost of the survey, discourages would-be respondents, and shrinks the sample size. It can be just as bad in business-to-business contexts. Is it any surprise that engineers get irritated by a 130-question e-mail survey from a leading technology company? One wag calculated that if 85,000 employees spent 26 minutes per survey, that would expend almost 37,000 labor hours, which he valued at $100 per hour. Result: The survey would cost nearly $4 million in engineering time.

9. The wrong customers respond

As a manager, do you really want to hear only from people who have nothing better to do than answer lengthy surveys? Chances are your most profitable customers ignore them. Surveying the wrong customer is a particular problem in the business-to-business arena. For example, many employees may use business software, but it is usually a handful of executives who make purchase decisions. When software vendors carpet-bomb customer lists with surveys, busy executives are apt to ignore them or delegate them to junior clerks. Most respondents will be people whose priorities don't necessarily match those of the managers making the decisions.

8. The right employees never hear the problems

If a customer does report a source of dissatisfaction, the information never reaches the people who can make it right for her. Instead, she becomes just one more data point in a marketing report. Indeed, how can frontline employees be expected to respond to the deluge of replies to all those questions? Even if they did have the time, they would probably be responding to the needs of the wrong customers. Customer feedback needs to come from customers with enough economic value to merit investing in solutions to their concerns. But customer anonymity is a hallowed principle of market research, so there isn't even the possibility of follow-up questions to probe root causes and possible solutions.

7. Many surveys are sales calls in disguise

Marketers have learned that prospects are less prone to hang up when asked a few initial questions than when they get an outright cold call. Result: Customers learn to mistrust anything that sounds like a survey. In fact, many firms commission surveyors to call customers but have no intention of fixing customers' problems or improving their experience. For instance, several auto manufacturers discovered they could boost their J.D. Power and Associates customer satisfaction ranking by calling all customers two weeks after they purchased a new car to ask if everything was OK. The manufacturers hired a vendor that had no way to take action on the feedback, because the real purpose of the call was to prime customers for a subsequent call from J.D. Power surveyors.

6. Survey scores don't link to a company's performance

If they did, Wall Street would be the biggest consumer of survey data. As it is, investors pretty much ignore customer-satisfaction reports. Bain research teams consistently find that the links between satisfaction-survey scores and customer behaviors that drive profitability or growth are tenuous at best. Detailed analysis of individual customers, for example, typically finds that 60%-80% of customer defectors score themselves as "satisfied" or "very satisfied" on surveys preceding their defection. Moreover, companies that achieve satisfaction scores of 80% or even 90% often experience no economic advantage from their apparent customer loyalty.

5. Plain-vanilla solutions don't fit companies' needs

Survey-research firms often push cookie-cutter solutions. But does a bank need the same information as an auto manufacturer? Predictably, research firm customers end up with useless generic data. Recognizing this, Pat Flood, CEO of Atlanta-based HomeBanc, had HomeBanc develop its own process: the company's closing attorneys give each customer a brief survey right after closing the loan-at the point at which any glitches in the process are fresh in the customer's mind. Almost 70% of customers respond, and the results are linked to the bonuses and promotions of each employee who interacted with that customer.

4. Everyone gets to make up their own rules

The new CEO of a global telecom company found that each country and business line in the company had its own satisfaction-survey vendors, its own methodology, and its own schedule for reporting results. If a division didn't like what it heard, the solution was simple: tinker with the methodology. (The CEO insisted on one set of basic rules for gathering and reporting information, but those rules were applied to make sense in the specific business settings.) Indeed, debates about survey methodologies run an amazing gamut. Consider the search for the best scale for measuring and reporting customer feedback. Some experts argue that a simple yes or no structure is best. Others advocate a 5-point scale where 1 means excellent, 3 represents neutral, and 5 means poor. Still others prefer to reverse that 5-point scale, making a score of 1 equivalent to poor, and so on. And some vendors don't provide a "neutral" box, forcing customers to choose between positive and negative.

3. Surveys confuse transactions with relationships

Gauging the quality of a single transaction is easy. But evaluating a relationship has to include every detail of the customer's experience—awareness, purchase, servicing, billing, upgrades, and so on. Companies rarely assess all these touch points. One large bank's first attempt at building a reliable relationship metric simply tallied up all the transactional satisfaction scores. Not surprisingly, these didn't tell the bank's managers much. Only when they began to home in on the goal of creating enthusiastic, delighted customers did they learn what mattered most to their core customers and focus on delivering those things to them.

2. Surveys actually dissatisfy customers

Who wants to be interrupted at dinner or badgered at work, just for a survey? A friend who bought a new car says he'd be more satisfied if the company just cut prices by the amount it spent on the multiple surveys he received. Most managers hire contractors who have little knowledge of customer needs or the sponsor's business. But it doesn't have to be this way. At Harley-Davidson, for example, customers are treated like family; they get phone calls only from recent Harley retirees (hired back part-time) who know the company and its products well and who are charged with listening closely to customers. Not coincidentally, these retirees generate deeper customer insights while also reinforcing the Harley culture and brand. At Southwest Airlines, President Colleen Barrett discourages large-scale surveys and instead insists that any employees who want feedback from a customer write a personal letter requesting that information and explaining what they intend to do with it. They must also write a thank-you to customers who respond, describing the actions that will be taken as a result of their feedback. Southwest also gains plenty of feedback from face-to-face interactions with customers.

1. Manipulation wrecks surveys' credibility

Think of your last visit to a car dealership: you probably saw an enlarged satisfaction survey on the wall, with all the top boxes suggestively checked. Turns out this boosts dealerships' ratings. Dealers even get salespeople to pressure customers for top ratings, often at the cost of providing free floor mats or oil changes in return. Some savvy customers have learned to play these games, too: they negotiate a low price and then offer a high satisfaction score to get an additional $500 off. After spending millions of dollars on satisfaction surveys, the car manufacturers have ended up with a system that irritates customers, turns salespeople into cynics, and fails to provide reliable scores to link satisfaction ratings with dealer profits or growth.

The truth is, meaningless measures of customer satisfaction don't help companies grow. What a business needs isn't satisfied customers but passionate advocates who sing the company's praises and who wouldn't dream of defecting to the competition. Gather regular, systematic, timely data about customer advocacy and make that information available to your operating managers right away.

The good news is that you can gather this data with as little as one question—I call it the "ultimate question"—and that's "How likely are you to recommend this company to a friend or colleague?" Ask this question regularly and systematically, then score the results on a 0-10 scale. You'll find out how many of your customers are loyal promoters (9s and 10s), how many are just passively doing business with you (7s and 8s), and how many are detractors (everyone else). The key to growth? More promoters, fewer detractors. Keep a close watch on your Net Promoter Score (NPS), the percentage of promoters minus the percentage of detractors, and you'll start to understand how many and which customers constitute your secret sales force. Companies such as GE have adopted NPS because it opens a window into how customers think about their organization. Call the profitable customers who are happy, and understand what you are doing right so you can do more of it; call those who are unhappy, and you'll learn what it takes to improve your offering and sustain profits, which is where the hard work begins. In any case, you can forget those cumbersome satisfaction surveys entirely.

Purchase the full text of this article from Harvard Business Online.