What is User Testing?

By Insight Platforms

• article
• explainer

• Customer Journey Mapping
• Usability Testing
• User Experience (UX) Research
• Customer Experience (CX) Feedback
• Remote User Testing
• Card Sorting

An Introduction to User Testing

User testing is a vital set of research methods that have been around for years. User testing involves research with users to find usability and user experience problems on websites, apps, or anything digital!

In fact, it doesn’t even need to be limited to digital environments. You can test the user experience of anything. For example: how easy is it to find a ticket machine in a train station? If you’re in one of the busy stations like London Kings Cross or New York’s Central Station – depending on which entrance you come in – it’s really not that easy. This is also user testing.

But for our purposes here, we will focus mainly on user testing of digital environments and experiences.  

A Word on User Experience (UX)

In order to talk about user testing, we first need to talk about user experience. User Experience is literally the experience of users when they attempt to do something: book a holiday online, search for an episode on a streaming site, or use a parking payment app. 

Nielsen Norman Group, the pioneers of UX, define it as follow:

User experience encompasses all aspects of the end-user’s interaction with the company, its services and its products.

It’s also important to note Nielsen Norman’s distinction between user experience and usability, which they define as ‘how easy & pleasant features are to use’. User Research is a research approach to understanding users, their needs and their experience. For example: what do users need from a car parking payment app?

More on User Experience (UX) here:

Definition of User Testing 

User testing measures both user experience and usability. 

User Testing is a set of user research methods for understanding the experience, finding insights and identifying issues so that developers and product managers can improve the experience. 

The aims of user testing are:

1. Find usability issues
2. Understand how users perceive and use the website or application – can they achieve their end goal? How easy was it? Were there any barriers? Do they trust the site i.e would they buy goods or services from the website?
3. Find bugs – naturally, user testing does involve some bug finding, but separate testing for bugs should be conducted and may not involve users 
4. Measure perceptions of a brand or business based on the website 
5. Increase conversions, sales, satisfaction, revenue and loyalty. 

User Testing types

There are three core types of user testing (moderated, moderated remote and unmoderated remote); there are also several other methods and tools that can help with user research (design-driven tasks like card sorting; feedback tools for polls and surveys; and user analytics tools such as heatmaps, click maps and screen recording).

Each approach has different advantages and different processes and tools involved. The three main types of testing involve participants, a test script, screening questionnaires where applicable, and screen recording software. The end result is a video of the user talking through the tasks and giving feedback on the website. 

We will cover the core user testing methods:

• Moderated testing 
• Unmoderated remote testing 
• Screen recordings
• Heatmaps & clickmaps
• Screen recordings
• 5-second tests

Recruiting the Right Users 

Before we delve into these methods, it’s important to remember to conduct participant screening to speak to the right users. There’s little benefit in speaking to someone unfamiliar with flying about how an airline checkout can be improved when they may have little idea of user needs. You can recruit from your opt-in email database, from social media, from research panels and participant recruiters as well as from user testing panels.

The most important aspect of user testing is the test script—this is what you write before entering a moderated session or the instructions you send to a user in an unmoderated remote session. It’s best to include a scenario to set the scene, tasks, questions and prompts to cover all ground. Don’t direct the user, ask them what they expect to happen. It’s also best practice when conducting research to avoid leading and loaded questions. 

A leading question is a question that encourages a particular answer. For example, ‘Overall are you satisfied with the experience of purchasing car parking on this mobile app?’ might encourage a positive response. A better question would be ‘Overall, how would you rate your experience of purchasing car parking using this mobile app?’ 

A loaded question is a question that has an assumption in it, for example, ‘Were you able to find a hotel of your choice on this website, or did you find it too difficult?‘ The last part isn’t needed and has an assumption about the experience. 

Moderated User Testing 

Moderated user testing—sometimes called lab (laboratory) testing—is a method of user testing that involves testing participants in a research environment (online or in-person). These sessions are similar to depth interviews in the sense that a moderator (researcher) and participant are in a (sometimes virtual) room for a research session. The moderator sets a scenario, sets tasks, and asks questions while observing the user on a website, system, or application. 

Unmoderated User Testing 

In the case of unmoderated user testing, there’s no moderator at all, and scenarios and tasks are set in advance. This is the most cost-effective method of user testing. It’s also the quickest in terms of turnaround time, but there are downsides: moderators are unable to dig deeper into a problem, and users may not always follow tasks or understand questions. But for some, it’s a helpful way of getting website feedback fast. Small businesses often opt for conducting unmoderated user testing to increase user satisfaction, conversions, or revenue. 

Heatmaps & Clickmaps

A heatmap shows you which areas of a webpage are getting the most attention from users. It shows how far users have scrolled on a page, for example, the top of the webpage which be a darker color as this usually gets the most attention, and the bottom (footer) of the webpage will be a lighter color, but results vary. Heatmaps can show you patterns of behavior as well as which parts of a webpage are performing better than others. 

Heatmaps can help you identify whether users are seeing important copy, CTAs (Calls to Action), or messaging or whether they are not seeing such important information. They can also help you reorganize your page so messages stand out more or test messaging to see if it’s getting attention. Clickmaps show where users click the most.

Questions heatmaps and clickmaps can answer: 

• We changed the CTA to blue – are more people noticing it?
• We added a new promotional banner to the homepage which promotes a sale on the site, are users noticing this version over the last one?

Screen recordings 

Screen recording tools record your visitors’ journey through a site. While there’s no audio, you can see the pages a user goes on and what they do on them. You’ll see what users click, how long they read, and how they move through your website. Screen recordings are a good way to observe user behavior on your website: how they use it, the journey they go through, what gets their attention, and the pages where they leave.

5-Second Tests 

Last but not least, the 5-second test is a quick and low-cost user research method. It involves showing participants a website for 5 seconds and then asking those participants what they remember about the website; sometimes, other questions are asked, too. You’ll get a quick understanding of what stands out on your website, what users think the key message is, and what users think you sell or offer.

User testing is an essential component of designing user-centric products and services. Through these methods, researchers can gain invaluable insights into user needs, behaviors, and preferences by directly involving users in the evaluation process.