What is Advertising Testing?

By Insight Platforms

• explainer

• Advertising Testing
• Advertising Experiments
• A/B Testing

Advertising is an effective tool for boosting brand awareness, shifting perceptions, and driving people to buy. It can truly make or break a brand. It is an investment, and it can be an expensive one, so you want to make sure that it’s an investment worth making. We’ve all seen ads and wondered how they got the green light (Pepsi, Dove, and many, many more).

There’s more competition than ever before for consumers’ attention. We are exposed to countless messages and impulses a day across computer, tablet or mobile device screens. So, it’s vital that your advertising stands out and does what you need it to do.

Advertising testing is a crucial step in creative development. It is a way of predicting the likely success of an advert or campaign, often before it is live (although not always), by reviewing whether it will meet objectives among the target audience.

Take our introductory course to learn the basics of Advertising Testing:

Depending on the method, it can also provide valuable feedback on anything that may need to be tweaked or improved to ensure the best impact. Additionally, if the ad or campaign performs well, it provides much-needed confidence in the creative direction.

How Can You Test Advertising?

There are lots of different approaches to advertising testing. Here are a handful of the most common; pre-test surveys, A/B testing, qualitative testing, and biometrics.

Pre-Test Survey

Pre-test surveys analyse the response to adverts before they are launched. With this method you recruit a number or research participants to take part in a survey. Participants are shown a creative (or number of creatives) and then asked a series of follow up questions to understand how they respond.

For this method, you should have at least 3 adverts to test, so that you can compare between them. There are 3 main designs for this approach.

Monadic Testing

This method splits out your sample into matched groups (cells), and shows the participant in each group one creative. You then compare the results between the groups to see which creative performs better on key metrics.

This is a fair way of asking about each advert, but requires a larger sample to reach a robust base size for each group. Also, you will need to ensure that the groups are matched on any key attributes, to ensure that any differences between the groups are because of the creative and not because of personal differences (i.e. different genders or age groups may respond differently to an advert).

Sequential Monadic Testing

This method presents an advert to a research participant and asks them follow up questions. When they have finished responding to one advert, they are they shown another. The adverts should be randomised in order, so as not to bias one advert over another.

This is a more efficient use of sample, as you don’t need to match groups, and can ask fewer respondents. However, the survey will take longer to complete this way, which could create fatigue, less reliable data (with straight liners) or drop out. Asking fewer questions that are more focused can help counteract this. It is also best to limit the number of adverts shown to 2, to avoid this.

Comparison Testing

This method shows two or more adverts simultaneously and asks respondents to answer follow up questions with them side by side. This creates a trade off scenario, where research participants will select one advert as more x than the other(s).

Again, this is a more efficient use of sample, making it cheaper. However but if adverts are similar, the results are not likely to vary much, making it challenging to find a winning creative. It can also feel more laborious to take part in, dependent on the number of adverts being tested. Finally, it may not draw out specifics or nuances, as participants will be less likely to give all creatives full attention.  

A/B Testing

This approach is widely used for digital analytics to test websites, apps, and digital marketing, but it can also be used for advertising testing. A/B Testing lets you test adverts simultaneously while they are live. This approach is particularly good for direct response adverts.

Two (or more) adverts are launched at the same time, to a smaller, matched audience, and you monitor which has more engagement. Digital advertising is particularly well suited to this approach, as you can track CTR or conversion rates.

That said, it can also work for other media. For instance, TV or radio adverts can be tested regionally to see which has the highest yield. This approach cuts out market research, by getting a clear understanding of how the adverts are impacting your margin. However, it will not give you the ‘why’ behind the ‘what’. You’ll be able to see that Advert A is better at converting, but you won’t be able to see creatively what drives that. Also, if any tweaks are needed to be made to make it work even harder, this will not be flagged through this method.

Qualitative Testing

This approach can either be through traditional face-to-face focus groups or using online qualitative tools. This approach can help you really drill down into what works and what doesn’t about an advert. Online qualitative tools can also help you get granular feedback at scale, meaning you can group themes, or look at subsets of audiences.

The upside of this is you can be very specific at an executional level, which can allow you to optimise creatives. However, if you want to be able to select a ‘winner’ out of the different adverts you have, it may not provide as clear cut an answer.

Biometric testing

Biometrics are used to understand physiological reactions to stimulus. There are range of different approaches (you can learn more about them here), but the most common of these are eye tracking and facial coding from expressions. These can be really useful tools for advertising testing, as they move beyond what people say they think, and get to the heart of subconscious feeling and behaviour.

Eye Tracking

Eye tracking is useful for all visual stimulus. It monitors where people look and for how long, to understand attention. To learn more about this approach, read our introduction What is Eye Tracking for Market & User Research? We also have a great webinar from Lumen that explores How to Win in the Attention Economy, which you can watch on demand here.

This approach is great at seeing what elements of a creative are standing out and grabbing attention, and can be benchmarked against other adverts in the same media or category. However, it is best when coupled with a degree of questioning to understand what has explicitly resonated, or to see what brand or messaging was recalled after exposure to the advert.

Facial Coding

Facial coding is a way of measuring the different emotions elicited in a research participant by coding their facial expressions. Like eye tracking, this approach is mainly used for visual stimulus. A camera (usually a webcam nowadays) captures changes in ‘micro-expressions’ on a viewer’s face as they watch an ad, a trailer or even longer-form video.

It can show whether someone is actually engaged with your creative, whether the humour is actually funny, or if they’re bored. This is particularly useful if your campaign is trying to create an emotional response, though is perhaps less relevant for tactical, direct response ads.

What Advertising Can You Test?

Advertising testing can happen at any point in the creative development process.

You can test messaging, copywriting, images, videos and audio. You can test an initial campaign concept, mocked up ads, right through to finished adverts. You can also test a full range of media types; TV, press, digital, radio, OOH, social, even door drops.

Just remember that if the ad is unfinished, you need to inform the research participants so that they are answering based on what they predict the finished result will look like, rather than telling you it looks amateur.

What Does Success Look Like?

Laying down expectations is a key part of the process. What do you want to achieve with your advert and among who? Remember that advertising can only do so many things at one time, so try to keep your objectives focused.

Here’s a list of some of the key types of questions you may want answered:

Creative Questions:

• Does the campaign stand out from others?
• How appealing is the advert?
• Is the advert memorable?
• Do people pay attention to it?
• What do people like or dislike?

Messaging Questions:

• What message did the advert convey?
• Does it tell people something they didn’t already know?
• Does it make people think this brand is better than others?

Brand Questions:

• Does it make people take action (buy product, enter a competition, etc.)?
• Does the advert make people care about the brand?
• Does it change people’s relationship with the brand?
• Does it change perceptions of the brand?

Tips for Successful Advertising Testing:

1. Set clear objectives

Before you start, make sure you have it clear what you are trying to achieve with the campaign. Are you trying to build brand awareness, are you trying to drive a call to action? Making sure these are as clear as possible means you are able to pinpoint whether an ad delivers on these metrics.

2. Set your target audience

This may seem an obvious point, and it aligns with clear objectives. If you need to understand what a specific segment think about your ad, make sure you have enough of them in your sample to be able to get a read. Also make sure that you ask the key questions to define them (such as segmentation questions). If you have multiple target audiences, make sure the research design allows for all of those groups to be analysed, which may require boosting sample or setting quotas. 

3. Build in time to make amends

It may be that the ad you test is perfect and ready to go, but it is very likely that the research will unearth some small (or large) tweaks that it is advisable to make, so try to give yourself some buffer time between receiving results and fully launching the campaign to implement any necessary changes to ensure maximum success.

4. Decide what and when to test

It usually works better to have 2-3 adverts at minimum to test, so provide a point of comparison. Equally, it’s best not to throw every single idea at a test. This will make testing more time consuming and more expensive, so have a shortlist.

Also, consider at what point to test. Do you need input at concept stage to drive development, or is it more useful to you when you are closer to a finished advert for final sign off and tweaks. In some instances you may want several rounds. Try to plan where you can.

5. Choose the right methodology

Think about the level of granularity you need in order to feel confident about launching your advert. Is it more important to understand multiple different target audiences or to get a diagnostic view on what is working and what isn’t. If it’s the former, a survey might be better suited, whereas in depth specifics are better acquired through a focus group.

Further resources:

Author