Digital Transformation
cancel
Showing results for 
Search instead for 
Did you mean: 

Lies, **bleep** lies, and statistics – be careful who you measure….

mtfitzgerald

Net Promoter Score is our primary customer experience metric. In the IT industry and at a worldwide level, 30 is a good score, 40 is a great score, and 50 is truly world class. Here is a guessing game. What is General Motors’ NPS score in their own survey of car buyers?

 

Before answering that, let me say a few words about phantom improvements. Have you read Freakonomics? The authors described what happened in Chicago when they started to measure individual teacher performance on the basis of their students’ results in standardized tests. There was a spectacular improvement in the results of the standardized tests. However, the researchers were able to discover that the teachers had found a variety of ways to cheat, varying from providing the answers to simply filling in the missing answers when students had not answered all the questions. The results improved, but nothing had actually changed.

 

Closer to home, I was the business leader in charge of the EMEA part of a major integration of two technology companies a few years back. Feel free to guess the names. At the same time I was dealing with a major customer (Vodafone) who was very unhappy with my company’s results in the Vodafone supplier survey. I managed to (legitimately) get hold of the other company’s own survey, called Voice of the Customer, that they had run with Vodafone. The results were spectacularly good. I went to Vodafone and tried to cut the conversation short by saying “I know we are awful. We are now merging with another company that makes you very happy. We will learn from them, and copy what they do. The Vodafone guy said “but… but… they are even worse than you!” I proudly produced the other company surveys and let him read them. His remark “but these two Vodafone people were not even involved in the project! Why did they get interviewed?”

 

Here is my point. The purpose of doing surveys is to learn about what is going on and improve it. If you bias surveys in any way, they are useless. In our industry, a common way of manipulating surveys is to avoid surveying people you know to have negative opinions. The motivation to manipulate arises when the person who selects who is interviewed is measured on the results of the survey. This leads to a golden rule of NPS. No individual must ever have variable compensation or bonuses that depend on the results of surveys over which they have any control whatsoever.

 

Back to General Motors. Their NPS is 95. The survey is done by sales people handing the survey to car buyers, and the sales person saying “Please fill out our survey. My job depends on you giving me a 10 on the recommendation question.” Are you surprised they score 95? (David Mingle of GM is the source for this information, in a public event last year.)

 

Maurice

 

Maurice FitzGerald - Customer Experience and Strategy consultant
maurice.fitzgerald@hnetplus.ch
  • IT Strategy and Leadership
About the Author

mtfitzgerald

Maurice FitzGerald ecently retired from his position as VP of Customer Experience for HPE Software after a career in hardware, software and services at DEC, Compaq and HP.

Comments
davjac

Maurice,

 

Your point about the rasion d'etre for VoC being improvement is well made but oft ignored.  Whether it is the result of vanity score chasing or insufficient time or attention , the value-added work of feedback - understanding and acting on results is all too often the weak link.

 

I too have experience of car dealers, in this case a BMW main dealer in the UK.  When a friend of mine gave an eight on the survey, the salesman went apopleptic and even called his manager to remonstrate and argue his case.  Unfortunately, I have yet to find a CEO of a car manufacturer (other than Rolls-Royce Motor Cars) who wants genuine feedback rather than an industry leading score.  I would love to be proved wrong.

 

Dave Jackson 

Labels
Events
28-30 November
Madrid, Spain
Discover 2017 Madrid
Join us for Hewlett Packard Enterprise Discover 2017 Madrid, taking place 28-30 November at the Feria de Madrid Convention Center
Read more
See posts for dates
Online
HPE Webinars - 2017
Find out about this year's live broadcasts and on-demand webinars.
Read more
View all