AI Agents Won’t Kill the Dashboard. They’ll Change Its Job.
By Caplena
- article
- AI
- Artificial Intelligence
- Customer Experience (CX) Feedback
- Data Analytics
- Dashboards
- AI Agents
Summarise with AI
A chatbot can answer every question you ask and still miss something important in your customer feedback. So how much should finding it depend on you knowing what to ask?
In my 2024 webinar on Insight Platforms, I asked if insights chatbots would replace dashboards or prove to be a gimmick. My answer was neither. Both had useful roles, though some of our expectations needed adjusting. As Caplena’s co-founder, I’ve spent the time since helping build these tools and seeing how people actually use them.
Here’s what I got right, what I got wrong, and what I’d ask a customer feedback agent to demonstrate today.
My 2024 Scorecard
What I got right: dashboards help people know what to ask.
You can scan a dashboard, notice a change, and decide where to dig deeper. Dashboards also evolved. Modern dashboards aren’t only a collection of charts; they include in-context insights and recommendations. On the other hand, chatting with an agent works well once you have a question. A blank input field gives you less to go on.
What I got wrong: who would use chat most.
We’d expected it to appeal mainly to people outside CX who only occasionally worked with feedback. Our usage data suggests that while users across the business use it as a copilot for instant answers, CX experts use agents as an exoskeleton to scale research, stress-test various hypotheses, and explore edge cases. Many conversations have over half a dozen questions & responses evolving a story in collaboration with the agent instead of exposing single metrics.
What became standard: numbers you can trust.
That reliability is now a baseline expectation in the RFPs we see. A fluent answer only gets you so far if the maths doesn’t check out. In 2024, our chatbot used an analysis engine to calculate results, and we worked hard to stop the language model from inventing its own numbers. Its tools were basic: counting topics, filtering rows. Learning from the questions our users asked, we kept adding more. Today the agent computes statistical significance, finds hidden segments and transforms entire columns on the fly.
What surprised me: how much more than “answering” agents could do.
Agents can monitor incoming feedback, investigate changes, and suggest improvements to topic collections. They can also help clean and enrich data or identify segments worth exploring. Recently, I asked our agent to analyse the quality of service from restaurant reviews by weekday. The agent realised the weekday wasn’t directly available in the data, so it created a smart column to extract the weekday from the date and then did an analysis on that. All within about 2min.
To make that change concrete, let’s revisit the airline reviews I used in the webinar.
From a Sandwich Question to a Dessert Alert
Back in the 2024 webinar, I started with a pressing research question: what did airline passengers think of the sandwiches? The system found a comment about a reindeer sandwich. Obviously, I needed to know which airline served it. The chatbot followed my curiosity, but I was still supplying the questions.
Now imagine an agent monitoring onboard food satisfaction. It flags rising dessert complaints on the Berlin–JFK route without waiting for you to develop a sudden interest in pudding. That’s your cue to open the dashboard. How many passengers mentioned it, over what period, and does the pattern appear on other routes? The agent starts the investigation, and you use the wider view to see what it found.
The dessert example is hypothetical, but the process is real. At Lufthansa Group, teams already combine conversational analysis with Caplena dashboards in a Customer Insights Hub used by over 1,000 employees. They can ask specific questions and explore the wider data in the same place.
That’s why I see agents and dashboards working together. The agent flags something worth investigating while the dashboard helps you judge what it means.
Caption: A Caplena dashboard comparing Heathrow and Amsterdam Schiphol airports, with driver analysis on the left
So what should you expect from the agent to make that partnership useful?
Four Tests for a Useful Customer Feedback Agent
An AI agent for customer feedback uses analytical tools to pursue a task over several steps, choosing what to look at next within the scope you give it. That could mean investigating a research question or monitoring an issue over time.
In my experience, the best AI agents for customer feedback analysis combine traceable answers, relevant initiative, and control over the underlying analysis. Test those qualities with your own data, where you know enough to spot a convincing answer to the wrong question.
1. Can I inspect and challenge its answer?
Make it show its working. If complaints increased, compared with when? Among all respondents, or only those mentioning food? Expect clear filters, sample sizes, supporting comments, and appropriate statistical tests. Three memorable complaints don’t establish the size of a problem, however quotable they might be.
Caption: Caplena agent analysis: key quantitative findings on food service in airline reviews
Keep the analytical rules intact. Weighting, exclusions, and segment definitions should remain consistent between chat and reports. And so should permissions. Asking a clever question must never unlock another client’s data.
Try changing a filter and asking the agent to explain how it affects the result. Then check the supporting comments. You should be able to trace its answer back to the feedback it’s based on.
2. Can it bring me something worth investigating?
Expect initiative. The best insights shouldn’t be prompted; a customer feedback agent should be able to surface evidence worth your attention and changes that matter.
Platforms like ours support both direct questions about feedback and continuous monitoring, with scheduled analyses,alerts, and proactive insights. Findings can reach Slack or Teams, while MCP connections make the analysis available through tools like ChatGPT, Claude, and Copilot.
Caption: Answers where people already work: a Teams chat in which Caplena summarises last week’s trending positive topics.
Monika Wirtz, CX Platform Owner at congstar, put the stakeholder need neatly: “Stakeholders don’t want dashboards; they want answers.” That’s compatible with keeping dashboards. A manager can receive an answer where they work, while an analyst keeps the wider view needed to investigate it. Everyone doesn’t need the same starting point.
For an ad hoc study, “initiative” might mean suggesting an overlooked segment or following a question through several comparisons. Ask what the agent would investigate next, and why. The explanation should make sense to the researcher who owns the study.
3. Can it take action, beyond analysing the data?
Expect it to finish the job. An answer is only half the work. Someone still has to set up the alert, build the chart, save the segment and tell the right team. An agent that can do those steps itself turns an insight into action in minutes, not weeks.
Think of a question that leads straight into a task. “Alert me when satisfaction with delivery drops in Germany.” “Save the customers who mention waiting times as a segment and chart their NPS by month.” “Send a short follow-up survey to the customers who complained about the new app.” The agent should create the alert, the segment, the chart or the micro-survey, not just describe how you could.
Ask the agent to act on its last finding. Then check what it created: what it does, who sees it, and how to undo it. If you can’t answer all three, it isn’t ready to act on its own.
4. Can it help improve the analysis itself?
Let it question the structure. Your topic collection might reflect last year’s priorities beautifully, but customers might have already moved on. An agent should help identify missing or overlapping topics and propose useful new ways to examine the feedback.
Keep changes reviewable. The same applies to cleaning, mapping, enriching, and structuring data. Ask what the agent proposes to change and what the consequences would be before applying it. Most importantly, an agent should never replace your source data, but rather create new columns augmenting your analysis potential.
Merging two topics could improve today’s analysis while changing the meaning of a year-long trend. Human-in-the-loop needs to mean control over decisions like that, with a clear record of what changed. The human shouldn’t discover the new methodology halfway through a client presentation.
The Calculation Can Be Right and the Conclusion Wrong
Back to our dessert problem. Suppose the airline adds a survey question specifically about food. Dessert complaints rise because more passengers now comment on their meals, even though the menu hasn’t changed. An agent could count every complaint correctly and still conclude the food got worse, unless it knows the survey changed.
That’s the bottleneck I see growing: context. Once the maths is reliable, good analysis depends on knowing what the data represents and which comparisons make sense. I explored this with Gregorio Uglioni on the CX Goalkeeper podcast. More and more, insights teams own the quality and context behind the answers.
A column called “Q14b” says little. A project description that just says “customer survey” doesn’t say much more. Give the agent the briefing you’d give a new researcher: survey changes, exclusions, relevant business events. For agencies, that includes the client knowledge that usually lives in a project manager’s head.
An agent can help document a change or flag a missing definition. But your team knows why the question changed and whether results are still comparable. That knowledge needs to reach everyone using the data, including the agent.
Here Are My Bets for 2028. What Are Yours?
In the 2024 webinar, I made one addition to the technology hype curve. A “graveyard of ideas” for the technologies that never really find a useful job. Two years of progress later, I’m ready to make a couple more predictions. Conveniently, you’ll have this article to hold me to them.
1: Insights teams move from gatekeepers to enablers.
By 2028, insights teams will no longer be the gatekeepers of insight. Instead, they will become the enablers who build and govern the system that lets everyone else get answers through agents on demand.
Getting there takes a step change in tooling. One that lets other teams query agents ad hoc, while giving a small insights team the means to monitor quality at scale and step in when results are misinterpreted or unfounded.
2: Fully synthetic research ends up in the graveyard.
Synthetic responses will find useful jobs. But the idea that they will replace surveying and interviewing real customers is the one I’d bury.
What are your predictions? Think about the work you did this week. Which task would you happily hand to an agent, and which would you want to keep doing yourself? Let’s compare notes in 2028.
