What is User Testing?

An Introduction to User Testing

User testing is a vital set of research methods which have been around for years. User testing involves research with users to find usability and user experience problems on websites, apps or anything digital really!

In fact, it doesn’t even need to be limited to digital environments. You can test the user experience of anything. For example: how easy is it to find a ticket machine in a train station? If you’re in one of the busy stations like Leeds, London Kings Cross or Berlin – depending on which entrance you come in – it’s really not that easy. This is also user testing.

But for our purposes here, we will focus mainly on user testing of digital environments and experiences.  

Definition of User Experience (UX)

User Experience is literally the experience of users when they attempt to do something: book a holiday online, search for an episode on a streaming site or use a car parking payment app. 

Nielsen Norman Group, the pioneers of UX, define it as follow:

User experience encompasses all aspects of the end-user’s interaction with the company, its services and its products.

It’s also important to note Nielsen Norman’s distinction between user experience and usability, which they define as ‘how easy & pleasant features are to use’.

Definition of User Research

User Research is a research approach to understanding users, their needs and their experience. For example: what do users need from a car parking payment app?

Definition of User Testing 

User testing measures both user experience and usability. 

User Testing is a set of user research methods for understanding the experience, finding insights and identifying issues so that developers and product managers can improve the experience. 

The aims of user testing are:

  1. Find usability issues
  2. Understand how users perceive and use the website or application – can they achieve their end goal? How easy was it? Were there any barriers? Do they trust the site i.e would they buy goods or services from the website?
  3. Find bugs – naturally user testing does involve some bug finding, but separate testing for bugs should be conducted and may not involve users 
  4. Measure perceptions of a brand or business based on the website 
  5. Increase conversions, sales, satisfaction, revenue and loyalty. 

User Testing types

There are three core types of user testing (moderated, moderated remote and unmoderated remote); there are also several other methods and tools that can help with user research (design-driven tasks like card sorting; feedback tools for polls and surveys; and user analytics tools such as heatmaps, click maps and screen recording).

There are different advantages to using each approach, and different processes and tools involved in each. We will also explore depth interviews as well as website and exit intent surveys as methods of user research and user testing. 

The three main types of testing involve participants, a test script, screening questionnaires where applicable, screen recording software and the end result is a video of the user talking through the tasks and giving feedback on the website. 

We will cover the core user testing methods:

  • Moderated testing 
  • Moderated remote testing
  • Unmoderated remote testing 
  • Other tools and methods: screen recordings, heatmaps and AB testing, card sorting, tree testing and surveys / polls.  

Moderated In Person User Testing 

User Interview - Insight Platforms
Photo by Amy Hirschi on Unsplash

What is it?

Moderated user testing – sometimes called lab (laboratory) testing – is a method of user testing which involves testing participants in a research environment such as a user testing lab or market research viewing facility (standard meeting rooms can also work well).

How to do it

Moderated testing sessions are similar to depth interviews in the sense that a moderator (researcher) and participant are in a room for a research session. 

The moderator sets a scenario, sets tasks and asks questions while observing the user on a website, system or application. 

Equipment and tools 

You’ll need: 

  • Screen recording software such as OBS studio which is free
  • Cameras for the participants’ faces and for recording the screen if you’re testing on smartphone or tablet. 

Top tip: iOS has a screen mirroring feature: you can mirror it to a laptop using software such as Reflector


Moderated Remote User Testing 

What is it?

Unmoderated remote testing is very similar to moderated testing in that a moderator leads the session.

However, remote moderated testing takes place over the web – the user isn’t in the room, and may be at home or in an office. The session is still very much led by the moderator, but moderated remote testing is much more cost-effective than hiring viewing facilities or travelling long distances for researchers to be with participants.

UserTesting screenshot - Insight Platforms

Copyright: Usertesting 

Usertesting allows you to schedule live remote moderated testing on their platform with their panel or your own customers.

How to do it 

Using conference calling software, call the participant and ask them to share their screen. 

Record your screen (which can see the screen of the participant) using another screen recording software. 

Equipment and tools 

Screen sharing software such as Zoom, Skype or Google Hangouts. 

Screen recording software: OBS studio, Quicktime


Unmoderated Remote User Testing 

Lookback Self Testing - Insight Platforms
Source: Lookback

What is it?

The final type of user testing is where there’s no moderator at all, with scenarios and tasks set in advance.

This is the most cost-effective method of user testing, it’s also the quickest in terms of turnaround time, but there are downsides: moderators are unable to dig deeper into a problem, users may not always follow tasks or understand questions. But for some it’s a useful way of getting feedback on websites fast. It’s a brilliant way for small businesses to conduct user testing to increase user satisfaction, increase conversions and increase revenue. 

How to do it

On a remote unmoderated testing tool of your choice, set up a screening question, select your targeting (such as females age 30-40 in the UK) and create your test by providing your test questions as well as the test website URL.

The platform will automatically find participants who match your targeting criteria and then they will conduct the test while the platform records their screen. Platforms usually allow you to ‘tag’ the end videos so you can easily find key issues. 

Equipment and tools

Examples of remote user testing platforms include:

Recruiting the right users 

Remember to conduct participant screening to speak to the right users. There’s little benefit in speaking to someone unfamiliar with flying about how an airline checkout can be improved when they may have little idea of user needs.

You can recruit from your own website using tools like Ethnio, which has participant scheduling within the tool, and Hotjar. You can recruit from your opt-in email database, from social media, from research panels and participant recruiters as well as from user testing panels such as those on Usertesting

Tools that can help with recruitment:

The importance of test scripts 

The most important aspect of user testing is the test script – this is what you’ll write before you enter a moderated session or the instructions you’ll send to a user in an unmoderated remote session. 

It’s best to include a scenario to set the scene, tasks, questions and prompts to cover all ground. Don’t direct the user, ask them what they expected to happen. 

It’s also best practice when conducting research to avoid leading and loaded questions. 

A leading question is a question which encourages a particular answer. For example:

‘Overall are you satisfied with the experience of purchasing car parking on this mobile app?’

The question encourages a positive response, a better question would be:

‘Overall, how would you rate your experience of purchasing car parking using this mobile app?’ 

Similarly, ‘Did you find any problems when purchasing your car park ticket’ encourages the user to suggest problems. A better question would be ‘How did you find purchasing car parking using this mobile app?’.

A loaded question is a question which has an assumption in it, for example, ‘were you able to find a hotel of your choice on this website or did you find it too difficult?

The last part isn’t needed, and has an assumption about the experience. 

Example test script 

Here is an example of a remote unmoderated test script to give you a flavour of how to write a user test script. Please note: this isn’t a full test script, it’s just to give you an idea of how to write a test script. This test script is written for a moderated test. 

Thanks for agreeing to take part in this website test, remember to speak aloud during the test, providing feedback on the site as you work through the tasks. This isn’t a test of you, it’s a test of the website and we’d like your honest opinions. 

[SCENARIO] Imagine you’re looking or a flight from London to New York, consider everything you’d need to consider and include during the flight booking process. 

[TASK] Starting from the homepage, search for a flight on a date of your choice for yourself from London Heathrow to New York. 

On the search results page: 

[QUESTION] Do you have enough information to select a flight of your choice? 

[PROMPT] If the user struggles to select a flight of their choice – ask them to select any flight – PROMPT: ask why they were struggling 

On the flight checkout page

[TASK] Please continue to checkout using fake details, please do not use your real credit or debit card details or real personal details. 

[QUESTION] If you were seriously considering booking a flight to New York today, would you complete this purchase?


Other User Research Methods 

There are a number of other approaches that help uncover user insights, including including

  • Heatmaps
  • Clickmaps
  • Screen recordings 
  • AB Testing
  • Card Sorting
  • Tree Sorting
  • 5 Second Tests
  • Polls, NPS & Exit Surveys

Heatmaps 

What is it

A heatmap shows you which areas of a webpage are getting the most attention from users.

It shows how far users have scrolled on a page, for example the top of the webpage which be a darker colour as this usually gets the most attention, the bottom (footer) of the webpage will be a lighter colour, but results vary and heatmaps can show you patterns of behaviour as well as which parts of a webpage are performing better than others. 

Hotjar Screenshot - Insight Platforms
Source: Hotjar

How to do it and tools to use

Heatmap tools are simple to set up on a website by pasting HTML code.

Tools include:

Benefits 

Heatmaps can help you identify if users are seeing important copy, CTA’s (Calls to Action), messaging or whether they are not seeing such important information. It can help you reorganise your page so messages stand out more or test messaging to see if it’s getting the attention. 

Questions heatmaps can answer: 

  • We changed the CTA to blue – are more people noticing it?
  • We added a new promotional banner to the homepage which promotes a sale on the site, are users noticing this version over the last one?

Clickmaps 

Crazyegg Snapshot - Insight Platforms
Source: Crazyegg

What is it

Similar to heatmaps, clickmaps show where users click the most. 

How to do it and tools to use

Again, you can simply paste a line of HTML code onto your site. Heatmap tools such as Hotjar and Crazyegg also provide click map reports. 

Benefits 

The biggest advantage is that it helps you see which CTA’s are getting the most attention, this allows you to test different messaging. 


Screen recordings 

Sesisoncam - Insight Platforms.png
Source: Sessioncam

Screen recording tools record your visitors journey through a site, there’s no audio but you can see the pages a user goes on and what they do on them. You’ll see what users click, how long they read for and how they move through your website. 

How to do it and tools to use

As with heatmaps and click maps you can simply add a code to your website. 

Tools include:

Benefits 

Although audio is missing, it’s a good way to see the behaviour of your users on your website. How they use the site, the journey they go through, what gets their attention and pages in which they leave. It’s fascinating but doesn’t replace full moderated user testing, though it provides more insight than heatmaps. 


AB Testing

Optimizely AB Testing - Insight Platforms
Source: Optimizely

AB testing is an experimental process of testing a change on a website against the original (the control).

For example, if you had the hypothesis that changing the copy of a CTA might increase conversions, you would test the original ‘Click here’ against the test CTA which might be ‘Buy now’. Tests run until significance is reached (until the amount of traffic has visited both pages to provide a significant result). Test versions then either win or lose. 

Click here to view the AB Testing tools in the Insight Platforms directory:  https://insightplatforms.com/platforms/category/abtesting-optimisation-solutions/


Card Sorting

Card sorting helps redesign a website navigation. There are two types: open card sorting and closed card sorting. Tools include OptimalSort by Optimal Workshop and Userzoom.


Tree Testing

Tree testing is a kind of reverse card sorting, it involves testing the navigation you designed during card sorting works. Treejack by Optimal Workshop is a tree testing tool.


5 Second Tests 

The 5 second test is a popular, quick and low cost user research method. It involves showing participants a website for 5 seconds and then asking those participants what they remember about the website; sometimes other questions are asked too. You’ll get a quick understanding of what stands out on your website, what users think the key message is and what users think you sell or offer.

UsabilityHub provides a panel and pre-set questions for 5-Second Tests.


Polls, NPS and Exit Surveys

There are dozens of tools for capturing user feedback on site.  These include:

Leave a Comment

Scroll to Top