Ask how many customers you ought to interview and you will be given a number with great confidence. It will be five, or it will be twelve. Occasionally an older hand will decline to name a figure and say that you stop when you stop hearing anything new, which is wise and, on a Monday morning with a project plan to fill in, no help at all.
All three answers have real research behind them. All three have also been repeated so often, by so many people who never read the research, that they have come loose from what they were about. To choose between them it helps to go back and see what each study was trying to find out.
Five
The five belongs to Jakob Nielsen, and it was never about interviews. It comes from usability testing, in which you watch people attempt a task and note where they come to grief.
The rule rests on one measurement: a single tester, on average, runs into about 31% of the problems in a design. Add testers and the proportion climbs steeply, until five of them have found some 85% between them. It works because everyone in a usability test is walking the same path. If the checkout button is hard to find, most people looking for it will have trouble.
Even on home ground the rule is less solid than its reputation. One researcher took sixty testers and drew random groups of five from them, over and over. Some groups found 99% of the problems. Others found 55%. Eighty-five is an average, and a study does not get the average. It gets one group.
Interviews are another kind of work. Nobody is following a single path through a single screen. You are asking about people’s jobs and habits and reasons, and a great many reasons belong to a small share of customers. Five people are unlikely to mention something that troubles one customer in twenty.
Twelve
Twelve has the better pedigree. It comes from researchers who did what nobody had troubled to do before: they conducted a large batch of interviews and recorded, one at a time, when new topics stopped turning up.
In the best known of these exercises there were sixty interviews. New topics had all but dried up by the twelfth, and the outline of the main themes was visible by the sixth. Others repeated the experiment on their own material, and when a review assembled 23 such tests, the point at which nothing new appeared fell between 9 and 17 interviews. Researchers call this point saturation.
So twelve is a respectable plan, subject to a condition that tends to fall off in transit. Most of those studies interviewed similar people about a narrow question.
There is a subtler catch as well. To have heard of a topic is not to understand it. In the one study that measured both, the list of topics was complete after nine interviews, while understanding each of them took between 16 and 24. Think of nine conversations about why customers leave. By the end you know that pricing keeps coming up. It may take as many again to learn that the trouble is the minimum number of seats, that it falls hardest on agencies, and that nobody notices until renewal.
Once you see what the twelve assumed, you can see what stretches it. The first thing is variety. If administrators and ordinary users have different problems, you are running two studies and must count them separately. In research across four sites, 16 or fewer interviews found the common themes at any one site, but the themes shared by all four took 20 to 40. Six per segment is a sensible opening bid. The second is breadth: “Why did you cancel?” runs dry sooner than “How does your team handle feedback?” One classic estimate from product development puts it at 20 to 30 interviews to uncover 90 to 95% of customer needs. The third is the conversations themselves. The more each one contains, the fewer you need, and one interview with good follow-up questions is worth several in which the customer says “it was fine” and you thank them for their time.
What neither can see
Five and twelve share a blind spot, and it takes one line of arithmetic to find it.
Suppose 5% of your customers are leaving because of a compliance requirement, and you conduct twelve interviews. That is and , and the answer is 46%. You are slightly more likely to miss the reason altogether than to hear it once.
| Share of customers with the problem | 5 interviews | 12 interviews | 30 interviews | 100 interviews |
|---|---|---|---|---|
| 20% | 67% | 93% | Over 99% | Over 99% |
| 10% | 41% | 72% | 96% | Over 99% |
| 5% | 23% | 46% | 79% | 99% |
| 2% | 10% | 22% | 45% | 87% |
The top row explains why topics seem to run out at around twelve. A problem shared by one customer in five is all but certain to have surfaced by then. Everything common has been said. What has not been said is everything rare, and that takes far more: about 59 interviews for a good chance of meeting a 5% problem, about 149 for a 2% one.
These are the best odds on offer. The formula supposes that you chose customers at random and that everyone with the problem mentions it. Volunteers are not a random sample, and a hurried interview can miss a difficulty the customer really has.
Knowing when to stop
Which leaves the old hand and her advice to stop when you stop hearing anything new. It is the best of the three answers and the hardest to obey, because “nothing new” is a feeling. Happily, researchers have converted it into a counting rule that can be applied while the interviews are still under way.
After each interview, note the themes it contained and mark the ones you had not met before. After four, count the different themes so far and call that your base. Say it is twenty. From then on, look at each new pair of interviews and count what they added. When a pair adds 5% of the base or less, which here means one new theme or none, you may stop.
In the authors’ own material, the rule ended the more uniform studies after six to eight interviews, by which time it had caught about 80% of the themes that the full set eventually produced. Where a missed theme would be expensive, be stricter, and stop only when three interviews in succession add nothing. And decide early how finely you are slicing. “Pricing” as one theme is exhausted quickly. “Seat minimum”, “annual billing only” and “currency” take longer.
All these numbers date from a time when every interview cost somebody an hour. AI interviewers have changed the price, and it is tempting to suppose they have changed the answer. They have not altered how quickly themes appear. In one early study of AI-led interviews, new themes dried up at about 25, which its authors describe as typical of studies led by people.
What a large round buys is whatever twelve interviews were never able to supply. At a hundred, you have an 87% chance of hearing something that affects 2% of customers. You also acquire a number worth quoting: “three of twelve mentioned pricing” might mean anything from about 9% to 53% of your customers, whereas “fifty of two hundred” narrows it to between 20% and 31%. Size has its own delusions, of course. If only your happiest customers reply, two hundred interviews yield a very precise portrait of your happiest customers. The sound order is a few long interviews first, to learn what the themes mean, and then many shorter ones to find the rare themes and measure them all.
So who was right? Everybody, about something else. Five is the right number for watching people use a screen. Twelve is right for finding what one kind of customer has in common. And the old hand, who refused to give a number at all, had the only answer that does not need one.
How Diaform helps
Diaform runs the interview as a chat. An AI interviewer asks your questions one at a time and follows up on vague answers, up to five times per question, which makes a round of 100 or 200 conversations practical. Each gets a summary with word-for-word quotes, and AI reports cover 5 to 500 completed conversations. It does not replace your first few long interviews and cannot repair a biased invitation list. Try AI user research with a 14-day free trial.
