Does NPS predict growth?

In December 2003 the Harvard Business Review published an article whose title was a promise: “The One Number You Need to Grow”. Its author, a consultant named Fred Reichheld, proposed that companies stop sending long satisfaction surveys and ask a single question. How likely are you to recommend us to a friend or colleague, from 0 to 10? Call those who answer 9 or 10 promoters and those who answer 0 to 6 detractors, subtract one share from the other, and you had, he argued, the best single predictor of a company’s growth. Our guide to NPS covers the mechanics.

Few management ideas have travelled so far on so little arithmetic. Two decades later the number has a slide of its own in most quarterly reviews, usually in large type: “NPS up 6 points, to 34.” There is a small round of applause. Occasionally somebody asks what the company did differently, and nobody is quite sure, and the 34 goes into the board pack anyway.

The slide makes two claims without stating either. One is that the number tells you where the business is going. The other is that six points is a movement. Both have been tested.

The promise, examined

The first surprise was in the original evidence. Reichheld’s charts set scores collected in 2001 and 2002 against revenue growth from 1999 to 2002. The growth had come first. That shows that successful companies have enthusiastic customers, which few would dispute. A forecast has to run in the other direction.

When a Dutch team later tested both directions, the difference was plain. Every customer metric they tried tracked current sales growth about equally well, and predicted future growth about equally poorly.

Nor did the score prove better than what it had replaced. The first independent test went back to three industries Reichheld had held up as showcases: airlines, life insurance and computers. A standard satisfaction index matched growth better than NPS in two of the three. A later study of 93 Dutch firms was kinder, finding that NPS did predict which customers stayed. The plain share of very satisfied customers predicted it slightly better.

The most favourable result of all came from a study that followed seven American sportswear brands for five years. Changes in the score predicted sales growth in the short term, though the level of the score did not. And even that carried a condition of some interest to software companies. It held only when the survey included everyone who might buy, not just existing customers. Most SaaS teams survey their current users, which is the version that showed no power to predict.

So the one number turned out to be one of several, no better than the others, and better at describing the past than the future. Nearly all of this work was done on consumer businesses such as airlines, banks and retailers. Nobody independent has run a large test on subscription software, and the most useful check is one you can run yourself: did the customers who scored you low last year leave more often than the rest?

Six points

The second claim is the easier to dispose of, because it needs no research, only a formula.

Part of the trouble lies in how the score is built. NPS takes eleven possible answers, sorts them into three groups and then ignores the middle one. A 0 counts the same as a 6. A 7 counts for nothing. If ten customers in a hundred nudge their answer from 8 to 9, the score leaps ten points, though hardly anything about how customers feel has changed. A recent review found that this way of counting adds extra wobble compared with a simple average of the same answers.

The larger part is ordinary sampling. Your survey reaches some customers and not others, and a different handful would have produced a different score. Because NPS is one percentage subtracted from another, the uncertainty in each adds up. There is a short formula for how much.

Suppose 100 customers answer. Half are promoters and a fifth are detractors, so the score is 30. The formula gives 0.153, or about 15 points. The honest statement is that your score lies somewhere between 15 and 45, which is a less satisfying thing to put in large type.

Now consider the applause. A company collecting a few hundred responses a quarter would need its score to move by more than ten points before anyone could say that something had happened. A rise of six sits comfortably within the range of chance. This is, moreover, the generous version. Checked against real ratings of 236 companies, the formula came out slightly too narrow. It also assumes that the people who answered are a fair sample of the people who were asked, a large assumption that we examine in nonresponse bias.

What it cannot see

Even a perfectly measured score would have its blind spots.

It records an intention. Researchers who followed more than 8,000 American customers for two years concluded that the intention to recommend, on its own, wasn’t enough to predict who would go on to recommend or to stay. It is one more instance of the gap between what people say and what they do.

It misses your real critics. Most negative word of mouth comes from former customers and from people who never bought at all, and a customer survey doesn’t reach them.

And it does not travel. Across 18 experiments, people answering by telephone were more likely to choose the most positive answers than people answering another way, and the top answers are precisely what NPS counts. Countries differ as well: a study of 26 of them found large differences in how readily people pick the ends of a scale, a subject we pursue in cross-cultural survey bias. A score gathered one way, in one market, can fairly be compared only with itself.

A smaller job

None of this is an argument for abandoning NPS. It is an argument for asking less of it.

Used modestly, the score does two things well. It gives you a trend, provided you collect it the same way every time and respond only to movements larger than the margin of error. And it tells you whom to talk to, which may be the more valuable service.

A score cannot tell you what to change. The answer to “What’s the main reason for your score?” can. If the number is too unsteady to act on at 200 responses, the 200 reasons are the part of the survey you can use. Begin with the customers the formula throws away. When researchers matched the scores of 189 customers to what they later posted online, the ones in the middle posted a mix of praise and complaint. These are the people who could still go either way. Ask them what would make it a nine, and ask again when the answer is vague. Our guides to follow-up questions and analyzing open-ended responses deal with the two halves of that.

The reporting can change to match. State the score with its range. Decide before the survey how large a movement must be to count. Show how the answers were spread from 0 to 10, so that everyone can see whether the detractors are sixes or zeros. And keep the score out of bonus plans, since a number that moves ten points by luck mostly rewards luck.

So what did the company with the 34 do differently? Quite possibly nothing. A six-point rise on a few hundred responses is what you would expect to see, about as often as a six-point fall, when nothing whatever has changed.

That does not make the survey a waste of time. It means the valuable part was never on the slide. The number is a rough gauge of how customers feel, of no use as a forecast and blind to small movements. Collect it the same way each time, show the range, and save the applause. Then read what the customers wrote underneath.

How Diaform helps

Diaform asks the NPS question inside a short chat, then asks for the reason and follows up when the answer is vague, as many times as you allow. Every conversation gets an AI summary with sentiment and word-for-word quotes, plus the full transcript. It will not repair the statistics. Your margin of error still depends on how many people answer, and a score collected in a chat may not match one from an email form. Try it with a 14-day free trial.