In a Swedish supermarket, some years ago, shoppers were invited to taste two jams and say which they preferred. It looked like any tasting stand. A spoonful from one jar, a spoonful from the other, a finger pointed at the favourite.
Then the researcher offered a second taste of the chosen jam and asked what was so good about it. The jars, however, had been rigged. Each had two compartments, and with a turn of the wrist the shopper was handed the jam they had just rejected.
Most noticed nothing. They tasted the jam they had turned down a minute earlier and explained why they liked it best. It was less sweet, said some. The texture was nicer. It would go well with breakfast.
If people can account, fluently and in good faith, for a choice they never made, it is fair to wonder what we are hearing when we ask a customer, “Why did you choose us?”
Nobody was lying
There were 180 shoppers, and they tried teas as well as jams. Between them they caught about a third of the swaps. Even when the flavours were as far apart as cinnamon-apple and bitter grapefruit, no more than half the switches were detected.
One would like to think that an invented explanation gives itself away by sounding vaguer, or more hesitant, than a real one. The same research group looked into it. In a related experiment with photographs of faces, they compared the reasons people gave for choices they had really made with the reasons they gave for choices that had been swapped, and found very few differences between the two. A made-up reason is as detailed and as confident as a true one.
None of these people was dishonest. Psychologists call what they were doing confabulation: filling a gap with a plausible account and believing it. The shoppers experienced their explanation as a memory. So will your customer.
The jam has an ancestor, which shows where such explanations come from. In the 1970s, shoppers were shown four pairs of nylon stockings laid out on a table and asked which was the best quality. The four pairs were identical. Position settled it: the pair on the far right was chosen almost four times as often as the pair on the far left. Asked why they had picked theirs, not one of the 52 shoppers mentioned where it was lying. When the researchers suggested it, nearly all denied it.
The psychologists drew a conclusion that still unsettles people. When we explain a choice, they argued, we do not look inward and read off the cause. We reach for a common-sense theory of what usually causes a choice of that kind. Quality, feel and colour belong to everybody’s theory of how stockings get chosen. “It was on the right” belongs to nobody’s, and so nobody says it.
Other people’s influence is another cause we are inclined to miss. In a survey of 810 Californians, what people believed their neighbours were doing predicted how much energy they saved better than any of their other beliefs. They rated it the least important factor of all. So when a customer tells you, “It made no difference that my whole team was already using it,” that may be true. It is also exactly the sort of cause that people do not see.
Witnesses, not analysts
Consider what follows when a customer says, “We chose you for the reporting.” It is a tidy, sensible reason, and it goes straight into the win-loss notes. Next quarter the team invests in dashboards.
The customer may be right. They may equally be describing the feature they have come to like since, projected back on to a decision taken for another reason. They cannot tell the difference, and from the transcript neither can you. A wrong reason is expensive in a particular way: it sends you off to build more of what the customer named, while the thing that won the deal goes unremarked.
It would be wrong to conclude that customers are always mistaken. Where the cause is obvious and also sounds like a good reason, people report it correctly. Somebody who left because a competitor halved its price will tell you so, and may be believed. The research, too, concerns quick, small choices, jam and stockings, and not companies buying software over several weeks. A stated reason is not worthless. It is a lead, and leads need checking.
Happily, the field that produced these warnings also worked out where people’s accounts can be relied upon. People can reliably report what they were paying attention to at the time. The errors begin when they are asked about something they never noticed, for then they must work the answer out, and what they work out feels to them like a memory.
Customers, in other words, are good witnesses to events. They can tell you what happened and in what order. They know what they did, what they had tried before, what went wrong, who else was in the room and what they said to their manager. Often they can recall how they felt at one particular moment.
They are poor analysts of their own minds. They cannot tell you how much each factor weighed, what they would have done had something been different, what they will do next year, or whether a thing influenced them. There is a partial exception. In a study in which people chose between rental homes from a table of nine features, most could say fairly well how much each feature mattered to them, though some were far out. But real purchases seldom arrive laid out as a tidy table.
The rule that follows is brief. Trust the story, and doubt the explanation.
The day it happened
If people remember events and guess at causes, the way to interview them is to ask about events.
There is direct evidence that it helps. In a later version of the face-swap experiment, an interviewer first took each person back through the moment of choosing and had them describe it step by step. After that, they caught 80% of the swaps, where people ordinarily miss most of them. It is a single study, but it agrees with what experienced interviewers have done for decades. The critical incident technique asks for one real event in detail. Jobs-to-be-done interviews walk a buyer along the timeline from the first thought to the first use.
In practice each “why” becomes a question about a moment. “What was the most important factor?” becomes “What did you compare us with? What did you tell your team?” “Why did you cancel?” becomes “When did you last use it? What did you use the week after?” “Did the price influence you?” becomes “When did price first come up, and who raised it?” Here is how it sounds with the customer who chose you for the reporting.
Interviewer: Why did you choose us over the other tool?
Customer: Mainly the reporting. Your dashboards are much stronger.
Interviewer: Take me back to the week you decided. What were you working on?
Customer: We had a board meeting coming up, and I needed churn numbers by Friday.
Interviewer: What did you do first?
Customer: I started a trial with both tools on the Tuesday. The other one needed our engineer to connect the data, and he was out until the following week.
Interviewer: When did you first open our dashboards?
Customer: Honestly, after the board meeting. That week I just exported a spreadsheet.
Interviewer: What did you tell your boss when you asked for budget?
Customer: That it was the one I could get working without engineering.
The first answer would have sent the team to build better dashboards. The story points somewhere else altogether: the customer could set the product up alone, against a deadline, and the competitor could not offer that. The dashboards were opened after the decision, so they cannot have caused it. Note, too, that the interviewer asked “why” only once.
One why
The restraint matters, because pressing for reasons can do worse than produce a wrong answer. It can alter the opinion.
In one experiment, students tasted five strawberry jams and rated them. Those who simply gave ratings agreed fairly well with a panel of expert tasters. Another group was asked to analyse their reasons before rating. Their ratings barely agreed with the experts at all. Asked to explain themselves, they had constructed new opinions from whatever was easiest to put into words.
No study supplies a safe number of follow-up questions, so what follows is a rule of thumb. Questions about the story may continue for as long as the customer is remembering: “What happened next?”, “Who said that?” Stop when you begin to hear “probably” or “I suppose”. Questions about causes should stop after the first. That one “why” gives you the customer’s own theory, which is worth knowing. A second and a third inquire into things they never noticed, and they will answer all the same. It helps to listen for the change of voice. “On the Tuesday I started a trial” is memory. “We’ve always valued flexibility” is theory.
The final step is what turns a lead into a finding: set what the customer said beside what the customer did. Go on collecting cancellation reasons, but file each as the customer’s summary, and next to it record the last specific event they described and their usage in the weeks before they left. “Too expensive”, from somebody who had not logged in for six weeks, is a usage problem wearing a price tag. Our article on why customers cancel goes through the common ones.
Feature requests want the same treatment. “We’d upgrade if you had single sign-on” is a prediction, the weakest kind of answer there is. Ask about the last time the missing feature cost them something, and what they did about it. A request with a recent workaround behind it deserves far more weight when you prioritize than one that arrives with a confident reason and nothing else. Importance ratings call for the same caution. In one comparison, customers rated nearly everything as important, so that their ratings could not tell the leading factors apart, while importance worked out from their behaviour explained satisfaction much better.
The shoppers at the tasting stand were neither fools nor careless. They did what all of us do when asked to account for a choice. They looked for an explanation that made sense, found one, and took it for the truth.
Your customers will do likewise, with the same sincerity. They are excellent witnesses to what happened and unreliable judges of why. Ask about one real day, in order. Take the “why” as a lead, and count it as a finding only when what they did agrees with what they said. Otherwise you may find yourself, next quarter, building a better version of the jam they turned down.
How Diaform helps
Diaform’s AI interviewer asks your questions one at a time in a chat and follows up on vague answers. You write the questions, so you can open with a timeline question and choose how far it follows up on each, and every conversation keeps its full transcript and word-for-word quotes. It will not tell you whether a reason is true, since an invented reason sounds like a real one to an AI as well. Export to CSV, compare with your usage data, and start with a 14-day free trial.
