How accurate does a fortune-teller need to be to lure us into the belief that they have genuine abilities? If we want to believe, and hear what we’d like to hear, not much at all. A quantitative performance indicator should accurately reflect changes in business performance over time. Net Promoter Score (NPS) was devised over a decade ago and remains popular despite serious flaws.
NPS is calculated from responses to a 0-10 scale question in the form of: “How likely are you to recommend this product/service to someone you know?”. The data transformation uses two thresholds and subtracts the percentage of “detractor” (0-6) responses from “promoter” (9-10) ones, resulting in a score between -100 and +100. Given the arbitrary thresholds used, and what I have previously written about threshold-based metrics, you might not be surprised by what follows.
To be concise, I will focus on one situation, and illustrate that NPS is especially flawed in a competitive market where customers have viable alternatives. How can the metric shift in response to a product/service change that is disliked by all of your customers? By indicating an improvement, obviously:
Here, I dropped the assumption that customer sentiment moves uniformly and instead consider two distinct groups. The “brand fans” group represents those customers who are invested emotionally and will advocate for your offering. A universally disliked change will not move them as much as the “ordinary customers” who, in a competitive market, can and will switch to a competitor’s offering, as well as having a more elastic reaction to your actions in the market. The key here is that the denominator shrinks as disgruntled “ordinary customers” switch away, while the numerator for “promoters” shrinks rather slowly as it is dominated by the inelastic “brand fans”. In this situation not one customer exhibits a positive response to the whatever has been changed, however, through the threshold magic of NPS, that inconsequential detail is obscured in the metric.
I do not know of any managers that have actually adopted a “drive away detractors” strategy to maximise their division’s NPS (comment below if you do). The above example specifically assumes a competitive market wherein mistakes can drive customers away. As with all threshold-based metrics, other forms of rank-inversion can occur without customers actually leaving, such as the calculated NPS changing in a different direction than an average of the individual data points.
Yes, we absolutely need to consider the voice of our customers, but rather than abstract rating numbers, we should be asking them the crucial question of why they like or dislike our offerings. That query leads to useful performance metrics. As for NPS, it is one number we need to forget.
1. ^ As illustrated above, blindly “growing” NPS is not desirable, however, this was the original advice given to businesses:
One thing that is not clear in the article is why, after some really good examples of customer satisfaction numbers being manipulated, the solution is… another number. If the car dealerships in the article adopted NPS, would their customers not just demand a discount for a favourable response to the NPS question as they had done for the survey scores? The solution appears to be a non-sequitur to the problem described.
Latest posts by Marcin Kreglicki (see all)
- Chaos, Compliance and Excellence - June 18, 2019
- One Number You Need To Forget - April 2, 2019
- Thresholds: The “Wishful Thinking” Approach to Designing Metrics - January 27, 2018