The Messy Fridge Problem: What AI-Moderated Research Reveals That Traditional Methods Can’t

By Conveo

• article

• AI
• Artificial Intelligence
• Qualitative Research
• Conversational AI
• Video Research
• AI Moderated Interviews
• Ethnography

When I used to run in-home ethnographies by physically sitting in people’s living rooms and walking through their kitchens, I noticed something almost every single time: the house was immaculate. Not just tidy. Immaculate. People had clearly spent time beforehand making everything look presentable, and I used to think “I’m a researcher, not an estate agent”. But that’s what happens when a stranger comes to observe you in your own home. You put your best foot forward.

The messy fridge stays firmly closed. The pile of washing-up gets done. The Chinese takeaway containers from two nights ago quietly disappear.

I’ve been thinking about this problem for a long time because it goes to the heart of a fundamental question about how we gather research data. The presence of an observer changes the behaviour of the observed. In research, we call it the observer effect, though you don’t really need a technical term for it – it’s just human nature, we perform for each other, especially when we know we’re being watched. The question I kept asking was: what would we learn if we could remove that dynamic?

Watch Conveo’s AI-moderated qual in action:

Eleven Million Surveys and Still Not Knowing Why

Before I joined Conveo, I ran the customer experience programme for Starbucks across the US and Canada. We were gathering twelve million surveys a year at its peak. We had eight or nine Likert-type measures and an open-ended question at the end: “Is there anything about that experience you’d particularly like to tell us?”. The data had a real influence on the business. Store managers could see their individual scores, and leaders tracked trends across regions. It was a serious programme.

We could see when customer satisfaction was rising or falling. What we couldn’t reliably understand was why. We had our prescribed measures – they’d tell us something – and we had the open-end, which in surveys most people either ignore entirely or answer with a sentence or two if you’re lucky. And even when we did get something useful, it had a timing problem. The survey went out a couple of hours after the visit. By then, the immediate emotional response to whatever happened in that store had already started to fade. The sharp edges of the experience had softened. The things that really annoyed someone or genuinely delighted them were already beginning to blur.

We were capturing data at the wrong moment with the wrong tool, and kept wondering why we kept getting a limited view.

Research at the Moment of Consumption

When I came to Conveo, what struck me almost immediately was a methodological possibility that hadn’t previously been achievable at scale. What if you could capture the research not hours after the experience, but during it, or within seconds of it ending?

We ran a study to test exactly this. We sent 75 people in the US to quick-service restaurants: half to one they normally visited, and half to one they didn’t. We asked them to order their food, and then to start the interview immediately. Not when they got home or after they’d had time to process and reconstruct. Immediately.

What we got was qualitatively different from anything a post-hoc survey could produce. People described the state of the counter, their interactions with the staff, and the temperature of their food. All this was not from memory, but from direct experience, in the moment. And because we were using video, we were capturing more than words – we could see their expressions. We could see them take a sip of a drink and hold it longer than expected, or push a burger away after one bite. The multimodal analysis layer, which examines facial expression, tone of voice, and physical behaviour alongside the verbal response, picked up emotional signals that no survey scale could have captured.

When we ran the analysis, the findings were direct and actionable. The main pain points at the point of ordering were menu clarity issues, wait times, service friction, and confusion around pickup flow, particularly when there was no staff member at the counter to guide the process. On the product side, temperature was the number one complaint, followed by order accuracy and freshness. Every one of those findings was linked back to specific video clips from real customers in real moments. Not a statistical inference, but a person, in a restaurant, telling you exactly what just happened.

The Confessional Effect

Going back to the messy fridge. What we’ve found consistently with AI-moderated video interviews is that people are remarkably, sometimes surprisingly, open.

There’s a theory about why this is. When you’re sitting alone, recording into a device, you know you’re not really talking to another person, even if the AI moderator speaks in a human-like voice and adapts its questions in real time based on what you say. There’s something almost confessional about the experience. You’re talking to yourself, in a sense. The social performance dynamic that shapes how you’d respond to a human researcher is largely absent.

We see this most vividly in sensitive research areas. We ran a self-commissioned study on GLP-1 weight-loss medications, and the depth of disclosure was remarkable. People were talking with complete candour about their relationship with weight, the difficulties they’d had, and the details of their treatment. In the home ethnographies I mentioned, when we ask people to open their fridge and show us what’s inside, they do. The Chinese leftovers are there, right next to the fruit bowl. You get the actual picture of how someone lives, not the curated version they’d present to a human observer.

I want to be careful not to overstate this. There are situations where the depth of human empathy a skilled moderator can offer is genuinely irreplaceable, particularly in the most sensitive clinical contexts or in live ideation, where you need the energy of a group. AI moderation isn’t a replacement for those methods. What it is is a genuinely different methodology with distinct strengths, some of which are unique to it.

Behaviour that Words Don’t Capture

One of the unexpected dimensions of this work has been what the multimodal capability reveals beyond what participants say.

There’s a study we ran with Unilever around food preparation and mealtime behaviour. One of the striking findings was that brands and products appeared in shot and were actively being used, significantly more often than participants mentioned or acknowledged verbally. People would say they used certain things and then not name them, because they didn’t think to. They’d reach for a product without thinking about the brand. But we were capturing it, right down to the SKU level in some cases.

This gap between reported and actual product usage is a well-known problem in research, but what struck me was that we were now able to quantify it in context. In a dishwasher study, for instance, when you ask someone to show you everything they use to wash their dishes, they’ll hold things up and say “I use these”. They won’t necessarily say what those things are. We don’t need them to, because the analysis captures the product, the brand, and the specific variant. That creates the possibility of filtering by actual product usage rather than claimed usage, which opens up analysis that simply wasn’t possible before.

A New Methodology, Not a Substitute

I understand why some people in the research industry feel anxious about AI moderation. There’s a reasonable concern that it represents a creeping substitution for skilled human work. I don’t see it that way, and I think the anxiety misframes what’s happening.

The research that I’ve described is immediate, in-context, multimodal, and scalable across markets. It is not research that human moderators were doing before, and is now being replaced by AI. It’s research that couldn’t be done at scale before at all. You can’t put a researcher in a car with every customer as they eat their lunch. You can’t sit in eleven million Starbucks conversations. The methodology has unlocked something genuinely new.

The question researchers should be asking isn’t “will this replace what I do?”, it’s “what does this let us do that we couldn’t do before?”. And then: how do we use it – alongside in-person qual, quant, everything else in the toolkit – to put consumers at the heart of more decisions, more of the time?

The messy fridge had a lot to tell us. We just needed to find a way to open it.

Watch Conveo’s AI-moderated qual in action:

Author

Learn more about