Designing Your First AI-Moderated Study: A Practical Guide for Research Teams

By Cint

• article

• AI
• Artificial Intelligence
• AI Moderated Interviews
• Online Qualitative
• Survey Panel

This article is based on the recent webinar, “The Agentic Shift: How AI-Moderated Interviews are bridging the qualitative-quantitative divide”, with Bruno Patriota, Senior Product Manager at Cint, and Sergio Perdices, Founder of Whyser. Rewatch the entire webinar here:

If there’s one piece of advice we’d give to any organisation about to run their first AI-moderated interview (AIMI), it’s this: start with the business question you’re trying to solve. Not “let’s go talk to customers because we have to talk to customers”, but what is the business decision we are trying to drive? What do we actually want to learn?

Once you can answer that, the rest follows. You can determine whether an AI-moderated interview is the right method to capture that data. You can define the conversations you want to have and the participants you need to talk to. Being very focused, especially in your first study, makes a big difference in how your experience with the methodology will go. The AI is very good at executing and finding information, but it won’t figure out what you need in your company. You’re the one who has all the context of that company and what really matters to you.

Think of it Like Any Other Qualitative Study

All the things that would make a traditional qualitative study great carry over to AIMIs. The design principles are the same. The difference is that AI is great at execution, but the front end and back end of the research are even more important. Researchers play a fundamental role.

On the front end, that means defining what the study is and what strategy you want to pursue. Is this a discovery study where you’re trying to uncover something new, or do you have certain assumptions you want to validate and have participants discuss? Those are very different briefs, and your study design needs to reflect that.

On the back end, AI isn’t great at the storytelling aspect. Translating findings into company impact, socialising them with the right departments, creating that loop of benefit where insights change how the organisation operates, from product to design to marketing to strategy – that is still very human-driven. It’s one of the things that makes researchers different from AI: strategy thinking, understanding company context, and getting insights back to the business.

Get the Study Design Right, Because the AI Will Amplify It

This is something we can’t stress enough: you will get results as good as your input. Your design, good or bad, gets amplified. The AI follows the script you’ve set, and it’s excellent at execution. So if you’ve set up a bad respondent flow, you’re going to get a very bad respondent flow.

When you define your interview, think carefully about how narrow or broad you set the topics. This will directly impact the freedom that the interviewer has in real time to probe into something unexpected or to focus more on your set of predefined topics. A totally unconstrained adaptive probing approach can lead to inconsistent coverage across different respondents. But being too rigid defeats the purpose of having an adaptive conversation in the first place. Finding the right balance is the researcher’s job.

Choose the Right Modality

AIMIs can run as text, audio, or video, and the choice matters. For text-based interviews, keep them under five to seven minutes. For voice, five to twenty-five minutes is the sweet spot. Around ten to twenty minutes still feels like something a participant can do at any time. Once you go beyond thirty minutes, it starts to feel like something they need to schedule, which reduces convenience and can increase turnaround time.

One advantage of AIMIs is that you don’t have to bundle many topics into one long interview. You can run more tactical, focused studies and launch multiple studies in parallel. That’s a shift in thinking from traditional qual, where the cost and logistics of each study pushed everyone to cram as much as possible into a single session.

You can also combine structured inputs with conversational depth in the same interview. Including a rating or scale, a multiple-choice question, or a card-sorting task gives you a structured, comparable data point across participants, and the AI can then interview the participant around their response to capture the qualitative depth behind it.

Consider whether to allow fallback between modalities. If your research goal is deep, meaningful insights and it’s really just about getting good quality answers, then giving respondents the option to switch between voice and text means fewer reasons for them to stop your survey entirely. That helps protect the integrity of the study.

Be Transparent With Respondents

Respondents should know they’re entering an AI-moderated experience. Every time. Data shows this is beneficial for everyone in the value chain. It’s an advantage for participants to know what to expect: they can do the interview at their convenience, it will be an engaging and dynamic experience, they can speak freely, and it’s best to be in a quiet environment.

The number of times we hear about participants who find out last minute and then decide to drop off is something we want to eliminate. If we can set those expectations upfront, everybody wins.

For sensitive topics, you need to put a lot of thought into your agent briefing instructions. Think about it the same way you would brief another person to conduct the interviews for you: what would they need to know and take into account?

Invest in the Respondent Experience

This is probably the biggest piece of practical advice on top of everything else. The price points for AIMIs tend to be a little higher than standard surveys, primarily because it’s more taxing on the respondent. Which means a good respondent experience matters even more.

It’s no different from quantitative research in that sense, but it matters more now because you’re typically looking at a longer interview. You need to make sure respondents aren’t running into technical friction with camera and microphone permissions. You need to set expectations upfront about what you’re going to do with their data and why they’re here.

Essentially, make them feel reassured.

And the best way to find out whether your study works? Go take your own survey. Get yourself in their shoes. Trial it and understand if it’s set up correctly and working as you expected. That simple step will do more for the success of your first study than almost anything else.

The Common Mistakes to Avoid

To wrap up, here are the pitfalls we see most often with first-time AIMI studies: not testing your own survey before fielding it; designing a respondent flow without experiencing it yourself; underestimating how much the quality of the study design matters when AI amplifies everything; trying to cover too many topics in a single interview instead of running focused, parallel studies; and not being transparent with participants about what the experience involves.

AIMIs are a genuinely new methodology. We haven’t had a new methodology in a very, very long time, and it’s an exciting opportunity. But like any research method, the fundamentals still apply. Start with a clear question, design the study thoughtfully, and put the respondent experience at the centre of everything you do.

This article is based on the webinar, “The Agentic Shift: How AI-Moderated Interviews are bridging the qualitative-quantitative divide”. Rewatch the entire webinar here:

Author

Learn more about