Why your NPS is higher in Brazil than in Japan

In Japan, the companies that people like have a Net Promoter Score of minus 47.

The figure comes from a consumer survey across 18 countries, which asked people how likely they were to recommend companies they like, and companies they dislike. In Brazil the liked companies scored 54. By the usual reading, Brazilian firms are adored and Japan’s best-loved businesses are in crisis.

Nobody believes that, put so baldly. Yet the same reading is performed once a quarter in any company that sells abroad, when the country table goes up on the big screen with the markets ranked by score. Brazil is at the top and is congratulated. Japan is at the bottom, as it has been all year, and its office is asked for a recovery plan. It may be the office with the highest renewal rate in the company.

The table is not wrong, in the sense that those are the numbers customers gave. What it conceals is that a score is two things at once: an opinion, and a habit of expressing opinions. The habit changes at every border.

A unit nobody prints

Everybody has a manner with rating scales. Some reach readily for the top or the bottom. Others keep to the middle. Some will agree with any statement put to them. Were these manners scattered at random, they would wash out in a large group. They are not. The average differs from country to country, and widely.

One study put the same five-point questions to students in 26 countries, each in their own language. The questions concerned work values and choice of studies. No product was involved, so there was nothing to be pleased or displeased with. Even so, students in Mexico and India chose “strongly agree” almost three times as often as students in Japan. The Japanese students used the middle of the scale more than anyone.

Two cities report today’s temperature as 30. One uses Celsius and the other Fahrenheit, so that one is sweltering and the other below freezing. Nobody would rank the cities by the number. Scores from different countries are in much the same position, except that nobody prints the unit.

These are patterns across large groups. They help in reading a market’s score. They say nothing dependable about how one customer in Osaka or São Paulo will answer.

Nines and tens

Any rating would show such differences. Net Promoter Score magnifies them, by construction. It counts only nines and tens as promoters. A seven or an eight contributes nothing, and anything from nought to six counts against you. In a market where people seldom use the top of a scale, the promoters all but vanish, however contented the customers.

Look at the last two rows. Consumers in Japan and South Korea gave a negative score to companies they like.

The spread tells a second story. In Brazil, liked and disliked companies stand 137 points apart. In Japan the distance is 41. The Japanese scale is not merely lower but compressed, so that a five-point movement there probably means more than a five-point movement in Brazil. The figures come from one vendor’s consumer panel and should not be leaned on to the decimal, but they point the same way as the students.

The question itself adds a difficulty. “Would you recommend us to a friend?” asks about your company, and also about the customer’s friendships. In Japan and Korea, to recommend something is to take on a social risk should it go badly. One paper describes an investment bank where more than eight in ten customers intended to use the firm again, and yet fewer than a third gave a top score for recommending it. When the low scorers were telephoned, most gave personal reasons, such as not wishing to discuss money with friends. The account rests on the authors’ own client work, which makes it a hint more than a proof. It fits the pattern all the same. Many of the “detractors” in these markets are satisfied customers who simply do not recommend things. The score’s other weaknesses are examined in whether NPS predicts growth.

English for everyone

A common response is to simplify: send every market the same survey in English, and at least the wording will be identical. It is a reasonable thought, and it makes matters worse.

A study of students in 24 countries compared the answers people gave in English with those they gave in their own language. In English, the differences between countries shrank on nearly every question. An English-only survey does not show your markets more fairly. It makes them look more alike than they are. Nor does the effect run in a single direction. In other experiments, people rating their feelings gave stronger answers in a second language than in their first. Either way, the same person yields a different number in a different language.

An accurate translation does not settle things either, because the labels on a scale carry different weight in each language. In one older comparison, the Japanese word for “good” was judged stronger than the English “very good”. A scale that looks identical on paper can be tilted before anyone has answered.

The usual safeguard is weaker than it is taken to be. In back-translation, a second person turns your survey back into English and you compare it with the original. A review of one large European survey found that the method raised many false alarms and left many real problems hidden. Its flaw is that nobody in the process reads the translated text as a native speaker would.

Survey researchers use a team in its place, as the European Social Survey does, and a small company can borrow the approach. Write the source in short sentences without idioms, with a note on what each question is meant to find out. Obtain two independent translations, from two native speakers or from one native speaker and an AI draft. Have a native speaker who knows your product compare them, attending particularly to the scale labels. Then try the result on three to five customers, asking them to say in their own words what each question means. An AI draft is a sensible place to begin, but it still needs that person. In one test of survey questions translated into German, the AI model rendered “extremely” with a word that implies excess, which quietly alters what the top of the scale means.

All of these studies were carried out on students, employees and consumer panels, mostly some years ago. Nobody has repeated them on customers of business software, who are often at ease in English and may differ less. Part of the gap between Brazil and Japan may also be perfectly real. Expect the direction to hold in your own data, and the size to be smaller.

Reading a market

Scores are one way of misreading a market. Silence is another.

In one experiment, American and Chinese participants were each put through the same failed computer repair. The Americans tended to complain to the company. The Chinese participants preferred private responses, of the kind the company would never hear about. So when the accounts in one country send few complaints, that is not yet good news. Criticism there may travel by another route. The remedy is to ask, and to make criticism easy to give. “If you could change one thing, what would it be?” invites an answer as “What do you dislike?” does not, and our guide to open-ended survey questions has more wordings of the kind.

The answers then want careful reading. An opening thank-you or apology is good manners, and the message begins after it. A gentle request, such as “it would be helpful if the export were a little faster”, may be the strongest thing that customer will ever put in writing, and deserves to be treated as a complaint. It is safer to count how many people raised an issue than to judge how angry they sounded. And keep the original beside the translation, since a label of “neutral” on a translated, indirect answer can hide a customer who was far from neutral. That goes for machine-written summaries too, one of the limits discussed in how far you can trust AI summaries.

As for the dashboard, it needs redesigning. Compare each market with its own past. Suppose Brazil has hovered around 54 for a year and slips to 49, while Japan has hovered around minus 20 and rises to minus 14. The country table says Brazil is 63 points ahead. The history says that Brazil has had a poor quarter and Japan a good one, and the second reading is the one that tells you where to look. Small samples jump about of their own accord, so check any movement against several earlier periods before believing it.

Rank markets on what customers do. Renewals, seats added and logins need no translation. Ask about events: “What nearly stopped you from renewing?” concerns what happened, and events survive translation far better than degrees of feeling, with no scale for anyone to use differently. And where you need priorities, ask people to rank the options. In one study, ranking removed the differences in scale habits between countries. For the same reason, avoid statements that people agree or disagree with, which invite yes-saying, as our piece on agree-disagree questions explains.

The head of the Japanese office, asked for her recovery plan, might reasonably reply that her customers are staying, paying and adding seats. They are also, like most of their compatriots, disinclined to give anything a ten or to press it on their friends. A score is a number plus a habit, and the habit changes at the border. Take the country table off the big screen. The thing in need of recovery was the dashboard.

How Diaform helps

Diaform’s AI interviewer works in 32 languages, so customers can type or speak in their own. It asks your questions one at a time and follows up on vague answers, which yields specific events and requests in place of scores alone, and every conversation keeps its full transcript and word-for-word quotes, so a native speaker can check the original. It does not correct for scale habits. An NPS question in Diaform is as country-bound as anywhere else, and your translated questions still need a native review. Try it with a 14-day free trial.