Testing – Too hard basket?

ADMA Education has just finished running a series of nationwide in-house courses on a combination of digital and direct response marketing topics. Testing was one of the important areas covered and through class discussion; it was revealed how little was known about the topic, and therefore how little real testing was being carried out. It seems the market is polarised between those organisations for whom testing is integral, and those that just don’t  do it. Why?

The testers are typically organisations who have embraced database and digital marketing whereas those who are relatively new to either of these disciplines usually don’t or maybe only dabble occasionally. The other main reason is that not enough is known about how to do it properly – basically it’s all too hard!

For anyone who’s not convinced of the value of testing it’s worth browsing through some case studies.  A good link on a range of case studies involving email, landing pages, direct mail and multi-channel campaigns is here and the benefits are obvious when you see the uplift stats.

When it comes to testing, the top three barriers are:

• Inability to track response
• Not being sure what to test
• Uncertainty of how to structure it properly. 

Some thoughts on these:

1. Inability to track response – this is the biggest barrier. If you can’t track it you can’t test it. In a situation where there is a direct response, tracking is possible through techniques such as unique phone numbers or urls, or campaign codes which link the response to the source. Failing that, scripting in call centres and online data capture works too. Where you’re not in control of the point of sale or the sales cycle is really long, the problem becomes more acute – although not insurmountable. In these circumstances quasi-tracking is possible, which means developing some rules of thumb based on what you can measure and using them to estimate. It does mean your testing becomes a bit blunt, but it’s still possible to compare things like offers, timing, channels – which are extremely useful things to test.
2. What to test – there are some guidelines which hold true regardless of the channel used – for example every component of the message – the product, proposition, creative, offer, call to action, and in terms of the delivery – the timing, the channel itself, the format and of course the targeting or segmentation strategy. Within each channel there are variations. Email for example – if you want more opens you need to try different subject lines, delivery times and frequency. If you want more click throughs, you need to look at  copy, images, layout, offers and the CTA. Once you get to the landing page all the things which are important in the  email are still relevant but by then people are looking for a bit more info. So figure out what their FAQ’s are likely to be and test different supporting information bullets. They’re also wondering whether this organisation is credible or not, so test different “trust elements”.
3. How to structure it – you can tick the first two boxes – tracking and what to test – then fail horribly if you don’t structure it properly. It’s about keeping it accurate, simple and robust. Simplicity is harder than it seems! It’s tempting to test several things at the same time and it is quite possible but you can easily run the risk of not being able to unravel the response data – which puts you back to square one. Accuracy is mostly a case of ensuring that the test is representative, is as free as possible of factors which obscure or adversely affect the results, and involves the use of a “control” to act as a benchmark. When we refer to a “robust test” we mean how reliable is it. This is a combination of the previous two items and a third, which always seems to be the subject of much debate, the scale of the test. Basically, too few responses and the test becomes unreliable, but you can draw some useful conclusions with surprisingly low numbers.

This is just a sample of what this topic covers and a snapshot of some of the areas which will be addressed in our new Testing for Success course. This course covers all the topics above and more, such as how to attribute success to specific channels when the vast majority of campaigns involve multiple channels and customer touch-points.

Find out more on the course website.