Fraud, Fakes, and Filters: Quality Control in the Age of AI-Moderated Research

By Cint

• article

• AI
• Artificial Intelligence
• AI Moderated Interviews
• Online Qualitative
• Survey Panel

This article is based on the webinar, “The Agentic Shift: How AI-Moderated Interviews are bridging the qualitative-quantitative divide”, with Bruno Patriota, Senior Product Manager at Cint, and Sergio Perdices, Founder of Whyser. Rewatch the entire webinar here:

A bot can fake a lot of multiple-choice answers. But as of right now, it’s still considerably harder for that same bot to fake a live, adaptive, fifteen-to-twenty-minute voice conversation with probing questions that build on every previous answer. That single fact is one of the most exciting things we see about AI-moderated interviews (AIMIs) when it comes to data quality.

However, let’s not pretend this solves the problem of fraud. We think of fraud as a wall and a ladder, and try to raise that wall to make it harder for fraudulent or misrepresented participants to get through. AI moderation raises that wall significantly versus a standard survey, and video raises it further. But the ladder will also grow. 

We’re already seeing real-time AI assistant coaching tools being used in hiring interviews, where someone sits behind the candidate feeding them clues and responses. Those same tools will find their way into research if they haven’t already. And video camera filters have gotten very sophisticated these days as well.

So the reality is that this is a continuous process. As we keep raising the wall, eventually the ladder will start to climb, too. And as emerging technologies bring new ways to combat fraud, we also face new challenges to overcome.

What AIMIs Actually Make Visible

One thing that’s genuinely different about AI moderation is how much more visible poor quality becomes. In a traditional survey, a participant who is multitasking, distracted, or lacking the required knowledge can click through multiple-choice questions, and you might never know. In an audio or video interview, that same participant is much easier to spot. Throughout adaptive follow-up questions, it becomes obvious when someone is fabricating answers or giving low-effort responses.

We’ve also noticed that the context surrounding the interview matters more than you might expect. Because AIMIs are asynchronous and participants have the convenience of doing them at any time, that flexibility has consequences. Someone doing an interview in the middle of the night while others in the house are sleeping will speak in a lower voice. They’ll be less expressive, and they might not be as open to elaborating. If somebody is in a public setting, on a commute, or not in a private environment, they might not be willing to share certain things. These factors affect the quality of the insights you capture, and they’re unique to this format.

The Layers That Still Apply

Many of the things the industry is now doing to combat fraud still exist and still matter. When the question is “what are we doing about quality in AIMIs?”, the answer starts with: much of what you’re already doing today. All of the existing multi-layered checks around respondent verification, demographic validation, and open-end quality still apply. AIMIs are an additional tool in the research process, so the quality infrastructure you’ve built carries forward.

But new quality signals also become available. Can we verify it’s an actual human voice? Is the respondent in a suitable room? If somebody claims to be a 45-year-old woman in Dallas, is there a way for us to check that against what we can observe in the video? These are the kinds of quality checks we’re exploring and working to nail down.

Where the AI Interviewer Can (and Can’t) Help

The AI interviewer can do some things to protect quality in real time. It can try to refocus a distracted participant and prompt someone to elaborate. But it can’t compensate for recruiting the wrong participant. If you’ve recruited someone who doesn’t have the right profile or relevant experience, no amount of adaptive probing will fix that. 

There’s also the question of what happens when technical checks fail mid-interview. If your research goal is primarily about audio and the audio fails, your multi-layered quality approach takes a hit. But because you have probing questions mixed in, and you’ve been building context throughout the conversation, there’s always the possibility that you’ve already captured enough to answer the question you just asked. 

Having fallback mechanisms, allowing a respondent to switch between voice and text, for example, helps protect the integrity of the study and reduces the reasons for someone to abandon it entirely.

The Respondent Experience as a Quality Lever

One thing we found interesting early on is how many researchers wanted video not primarily for the richness of the data, but simply to confirm there’s a real respondent on the other side. In that way, video has become its own fraud barrier.

But video also introduces friction, and that is where setting expectations upfront goes beyond courtesy and becomes a quality issue. If a respondent finds out at the last minute that they’re entering an AI-moderated session with a camera, and they weren’t prepared for that, they drop off. We don’t like drop-off, so if we can set those expectations upfront, everybody wins.

Transparency builds trust. And trust is what actually drives the quality of the data. If you’re upfront about what will be recorded, how the data will be used, and how it will be shared, you create the conditions for participants to be open and honest. 

We’ve run experiments on the sequencing of these permissions, and the difference is significant. If you try to get microphone and camera permissions before explaining what the study is about, conversion drops sharply. But if you explain first, in plain language, the key questions around data use and sharing, and then ask for permissions, participants are far more willing to engage fully.

Looking Ahead

Fraud in research has always been an arms race, and AIMIs don’t change that fundamental dynamic. What they do is add meaningful new layers of verification and visibility that weren’t possible with traditional survey formats, while also introducing new challenges that the industry will need to stay ahead of.

The practical advice we’d give is straightforward: keep doing what you’re already doing on quality and fraud. Layer in the new signals that AIMIs make available, but be mindful that new technology brings new vectors for fraud. And above all, invest in the respondent experience, because trust is the foundation that makes everything else work.

This article is based on the webinar, “The Agentic Shift: How AI-Moderated Interviews are bridging the qualitative-quantitative divide”. Rewatch the entire webinar here:

Authors

Learn more about