A night in hospital is about as memorable as ordinary life gets. Yet when interviewers for an American health survey spoke to people whose records showed a stay during the previous year, one stay in every eight was never mentioned.
Nobody was concealing anything. The stays had simply gone, the short ones more often than the long. If a hospital bed can slip a person’s mind inside twelve months, it is worth asking what chance there is for a confusing settings page.
The question matters because every survey and interview about the past is at bottom a request for a memory, and memory is not a recording. It loses things and moves them about and, given time, rewrites them. Consider a customer who agrees to a short call three weeks after cancelling. She is friendly and clear. “Honestly, it came down to price. We found something that did the job for less.” The support log has another account: four tickets in her second month about a data import that kept failing, the last unanswered for nine days, and a competitor’s trial started the week after. Price appears nowhere in the record. Ask her again and she will give the same answer, because that is how she remembers it.
What goes first
Researchers have spent decades checking what people say against records of what happened, and the discrepancies fall into a pattern. Where the facts cannot be retrieved, people fill the holes with fragments and reasonable guesses, and the answer still sounds like a fact.
Forgetting is the simplest failure. The second is odder. People remember an event and misplace it in time, usually drawing it closer to the present. In a Swedish experiment, people were asked how many nights they had spent in hospital in the past month, and their answers were checked against the national registry. They reported 73 nights that the registry didn’t show for that month. Many had been in hospital, only earlier. A short time window is therefore not the safe choice it appears. Ask “In the last 30 days, did you contact support?” and you will collect the ticket from six weeks ago too.
The third concerns numbers. We like to picture a customer replaying each occasion and adding them up. Mostly they do no such thing. People count only when the events are few and each one stands out. For anything routine they consult an impression and convert it into a figure, and the conversion leaves marks. When victims of repeated crimes were asked how many times it had happened in six months, the answers bunched at 10, 12, 15, 20, 24 and 30: round numbers, and monthly rates multiplied by six. “How often do you use reports?” yields the customer’s idea of what sort of user they are. The real frequency is in your analytics.
Quickest of all to go is how a thing felt while it was happening, which is unfortunate, since that is so often what is being asked. Patients living with pain rated their symptoms several times a day and were later asked to recall them. Looking back over the same day, they matched the record well. Beyond the past several days, they had growing trouble remembering how they’d felt.
What moves into the vacancy is revealing. Once the sensation has gone, people answer from what they believe about themselves. “I’m not very technical, so setup was probably hard for me.” It sounds like feedback on your onboarding. It is a description of the customer’s self-image, and would have come out much the same whatever your setup flow was like.
The story takes over
Forgetting is half of it. In place of the details people keep a summary, and the summary obeys rules of its own.
The best-known demonstration is Daniel Kahneman’s. He and his colleagues asked students to hold a hand in painfully cold water, twice. One round lasted sixty seconds. The other lasted sixty seconds and then thirty more, during which the water was warmed very slightly, though it remained unpleasant. Afterwards the students were asked which round they would sooner repeat. Twenty-two of the thirty-two chose the longer one. It contained strictly more discomfort. It ended better.
From this and similar studies came the idea that we judge an experience chiefly by its most intense moment and by its ending, and hardly at all by how long it went on. The idea holds best for short episodes. Over week-long holidays it was not an especially good predictor of how people remembered the trip, although they still ignored the length and could not recall how individual days had gone. A three-week onboarding has more in common with the holiday than with the bucket of cold water.
Opinions are revised in the same manner. In a study that ran for nine years, people were asked to recall the political views they had given at the start. Their recollections matched their original answers poorly, and they believed they had been far more consistent than they were. We remember having always thought what we think now.
Set these findings beside the customer on the churn call. In three weeks she has chosen a competitor, explained the switch to her manager and told the story a few times over. What she offers the interviewer is today’s verdict, projected backwards: one cause, one clean turning point. Nobody has measured the accuracy of exit interviews, so this is an inference from the research and not a finding. It is, though, exactly what the research would predict, and it agrees with what is known about why customers can’t explain their decisions.
When to ask
If different things decay at different speeds, the right moment to ask depends on what you want to know.
The effort or feeling of a task should be asked about the same day, since feelings fade within days and belief takes their place. The details of a support contact last one to three days, the small ones going first. Friction during setup gets less than a week before the minor obstacles are forgotten or smoothed over. A rare and important event survives for perhaps three months, though it drifts in time. Reasons for cancelling are best collected at the moment of cancelling, because the story is rewritten as the customer moves on. And how often something happens is best not asked at all. Past a handful of events the answers are guesses, and you have analytics.
These intervals are a judgment, drawn from studies of patients, households and students. Nobody has run the equivalent test on people using business software, so shorten them for anything forgettable. And observe what they imply about a common habit. An onboarding survey sent on day thirty is not telling you where onboarding went wrong.
No wording can restore a memory that has gone. Good wording can stop a customer from inventing one to be helpful. The most useful change is to ask about a single occasion. “Usually” and “typically” request an average, and an average has to be made up. “The last time” requests something that happened. So “How often do you use the reporting feature?” becomes “Think of the last time you opened a report. When was that, and what were you trying to find out?” And “Why did you cancel?” becomes “What was happening on the day you decided to cancel?”
Give the period a starting point the customer will recognise, such as “since you upgraded” or “since your renewal on March 3”. In experiments, a landmark at the start of the period cut the number of older events that people pulled into it. You also have an advantage most researchers lack, which is that you know what the customer did. Say so. “On Tuesday you exported a report to CSV” gives memory something to hold. Walk through the occasion in order, facts before feelings. And let “I don’t remember” be a perfectly respectable answer, because pressing for detail a person does not have produces a plausible invention.
The verdict
It would be a mistake to regard a late answer as merely a bad one. Sometimes the memory is exactly what you need to measure.
Researchers once tracked how students felt before, during and after their spring break. It was what the students remembered afterwards, more than what they had felt during the trip, that predicted whether they wanted to go again.
Renewals, referrals and reviews work in the same way. A customer deciding whether to renew does not replay eleven months of sessions. She consults the story she has kept. That makes the story worth knowing, provided it is not mistaken for a record. The sensible arrangement is two surveys: one straight after an event, about that occasion, to tell you what to fix, and one at a quiet moment, before renewal perhaps, about the whole relationship, to tell you what the customer is likely to do. Overall satisfaction can be asked at any time, since there the memory is the thing you are after. The trouble begins when one is read as the other, which is one more face of the gap between what people say and do.
So which was it, the price or the broken import? Both, about different things. The import is what happened. The price is what she will tell the next person who asks about you. To mend the product, you needed to ask her in the second month, about that import, on the day it failed. To learn what the market is hearing, the call three weeks later did very well.
Memory keeps the verdict and lets the evidence go. It is as well to know which of the two you want before deciding when to ask.
How Diaform helps
Diaform is a link you can share or embed where the event happens, so the customer answers while it is fresh. The AI interviewer asks your questions one at a time and follows up when an answer is vague, and respondents type or speak with no account needed. One caution: a follow-up can jog a real memory or invite a confident guess, so set follow-up depth to low on questions about anything older than a few weeks. Try it with a 14-day free trial.
