How to Keep Focus Group Analysis Accurate and Defensible When Using AI
By BT Insights
- article
- Artificial Intelligence
- AI
- Qualitative Research
- Group Discussions/Focus Groups
- Transcription
Summarise with AI
A client asks a simple question during the readout:
“How many people actually said that?”
If your answer is a confident number with the quotes to back it up, you’re in good shape. If it’s a pause, followed by “the AI flagged it as a key theme,” you have a problem.
That’s the gap between fast analysis and defensible analysis, and the research on AI-assisted qualitative work shows why it matters. One study cited in a 2026 review of LLMs in qualitative research found that AI cut inductive coding time by 81% while holding about 80% accuracy. That’s not just time saved; it also means roughly one in five coding decisions could be off if no one checks.
At BTInsights, we see this play out across the focus group projects our customers run. When AI focus group analysis goes wrong, the cause almost always sits upstream of the analysis itself. That means a weak transcript, a mislabeled speaker, or a model left to guess at something the team already knew.
Below are the seven practices we’ve seen make the biggest difference, grouped by where they happen in the workflow.
What “Defensible” Actually Means Here
Defensible analysis is analysis you can show your work on.
Every finding should connect to specific participants and specific things they said. Counts should be correct. And if a stakeholder challenges a conclusion, you should be able to walk them from the insight to the quote to the moment in the session where it happened.
Focus groups make this harder than one-on-one interviews. You have six to ten people talking, sometimes over each other, reacting to multiple stimuli, each with their own demographic profile. There are more places for small errors to creep in, and AI will build on whatever it’s given.
With that said, you need to start at the source and work your way up.
Stage 1: Get the Raw Material Right
The transcript being the most important element is obvious, but something AI can easily get wrong (or, in the case of a bad transcript, increase the likelihood of mistakes).
1. Start with a high-quality transcript
A lot of teams default to the built-in transcription in Zoom, Teams, or similar meeting tools. They’re convenient. But they’re designed for business meetings, not research sessions, and the quality often isn’t there when you have crosstalk, accents, category jargon, brand names, or a participant who trails off mid-sentence.
Small transcription errors carry real consequences. A dropped “not” can flip a participant’s meaning entirely. A misheard product name can split one theme into two, or hide it altogether. By the time the AI summarises the session, those errors look just as authoritative as everything else.
If you’re choosing focus group transcription or transcription software for qualitative research, test it on your hardest audio, not your cleanest.
2. Make sure speaker labels are accurate
Who said something matters as much as what was said. If a comment from a 24-year-old first-time buyer gets attributed to a 58-year-old repeat customer, every segment comparison built on that comment is wrong. And unlike a garbled word, a misattributed speaker rarely looks wrong on the page. The transcript reads perfectly well. It’s just assigning the insight to the wrong person.
Whatever transcription or translation tool you use, it should do two things: label speakers accurately from the start, and make it easy for your team to review and correct those labels before analysis begins.
Keep in mind that this step takes minutes per session, but having to fix a misattributed segment after the client has seen it takes a lot longer.
Stage 2: Structure the Data Before AI Reads It
In our experience, this is where many teams skip ahead. It’s also where a lot of accuracy is won or lost.
The idea is simple: the less the AI has to figure out on its own, the more of its attention goes to actual analysis.
3. Link demographics to speakers directly
Most focus group projects come with a screener or participant grid: age, gender, region, customer status, and so on.
Upload that profile data directly and connect it to each speaker in the transcript. Don’t rely on the AI to read the conversation and infer who’s a lapsed customer or who lives in a rural area. That kind of inference is error-prone, and it spends reasoning effort on something you already know for certain.
With demographics attached, segment questions become straightforward. For example, “How did participants under 35 react to the new packaging?” is answered from known data, not a guess.
4. Break sessions into stimuli or chapters
Focus groups usually follow a discussion guide. There’s a warm-up, a section on current habits, then concept A, concept B, maybe an ad test.
Categorise those segments first, then run the analysis by chapter or by stimulus. This keeps reactions to concept A from blending into reactions to concept B, and it gives the AI a focused slice of conversation rather than a 90-minute transcript with everything mixed together.
It’s also just way cleaner reporting, and most clients want findings organised by stimulus anyway.
5. Keep the analysis scoped to your transcripts
Every finding should come from what your participants actually said and nothing else.
Make sure your tool lets you define the context for each analysis, meaning the specific transcripts from this study, and that it answers only from that material. You don’t want general background knowledge, assumptions about the category, or unrelated documents shaping the output. If a finding can’t be traced to the sessions, it shouldn’t be in the report.
Stage 3: Check the Numbers and the Evidence
6. Use a dedicated feature for counting
Language models can work through super complex reasoning problems, yet they still (for now) make surprisingly basic counting mistakes. Researchers have documented models failing to count the letters in a single word, and the same weakness shows up when you ask how many participants mentioned a topic across several long sessions.
That can lead to real mistakes, because counting questions are some of the most common in qualitative reporting:
- How many participants mentioned price?
- How many preferred concept B?
- How many brought up the competitor unprompted?
For these, don’t rely on a free-text AI answer. Use a tool with a dedicated counting or extraction feature that tallies responses participant by participant and shows you who was counted. That way, the number is accurate (and you can check it).
7. Link every insight to its supporting quotes
Every insight should come with the quotes that support it, and each quote should link back to the exact spot in the raw transcript (ideally with the recording alongside). This is what takes hallucination off the table as a worry. Instead of trusting that the AI got it right, you can see the evidence and confirm it yourself.
It also changes the readout conversation. When a client asks “how many people said that,” you can show them.
A Quick Pre-Delivery Checklist
Before findings go to a client, it’s worth a final pass on the basics:
- Transcripts reviewed, with key terms and brand names correct
- Speaker labels confirmed for every session
- Demographics linked to each speaker
- Analysis organised by stimulus or discussion guide section
- All counts produced by a counting feature, with participant-level detail
- Every headline finding backed by linked quotes
If each box is checked, you can consider your analysis both fast and defensible.
Defensible Analysis Starts Before the First Prompt
None of the seven practices above is complicated, and most take minutes per session. Together, though, they decide whether your findings hold when a client asks how many people said something, or wants to hear it in a participant’s own words.
At BTInsights, our interview and focus group analysis platform was built around these same principles. Transcription comes with speaker identification your team can review, and every finding links to its supporting quotes, the original transcript, and the recording, so any insight can be verified in a couple of clicks. The speed AI brings to focus group analysis only counts when every finding can be traced back to what participants said, so make the most of the steps we covered to create more accurate and defensible analyses.
