Lyris Connections Blog

The 3-Step Strategy for Effective Email Marketing Split Testing

Jun 09, 2014 by

Split testing is the marketers version of going to the gym. We all know we should be doing it, and we all know it’s only going to be good for us. So, why don’t we do it as much as possible?

At the gym

Well, there are lots of reasons. Perhaps your previous tests have failed to show statistically significant difference. Or is it that you’re not sure what to test any more? I am sure though, that we all agree that the most common barrier to effective split testing your email marketing is time (funnily enough, the single most common reason why people stop exercising). Our daily lives as marketers are often complex, full of noise and distractions. Testing can often feel like it’s just adding to that daily, hourly noise and with little return. If this feels familiar, I hope the following advice will help you to convert that noise into something a little more useful. Let me tell you, if you do, the results can be incredible. 

Step 1 – Create a plan for split testing

As with any part of your strategy, budgets, campaigns, and so on, you have to plan ahead and schedule. Split testing is no different. This is often where I find that most brands are set up to fail – because there is rarely a plan or strategy for any email marketing testing. Create a plan – it can be as simple as a couple of extra columns in your campaign planner. However you choose to do it, just be sure to make a plan and try to stick to it.

When you plan for testing, not only is it more likely to take place, but you start to think about the test-ability of all sorts of wonderful things: layout and images, personalization and segments, subject lines and “from” addresses, day of send and time of day, and much more.

Step 2 – Effectively evaluate your results and implement change

You carry out a few tests and you can see some differences in opens, clicks, conversions, and so on, but how do you know if you should roll out what you have discovered through testing across all your email marketing?

One important answer to this question is to evaluate the statistical significance of your tests. You have to prove that your tests results can be relied upon to when implementing what you have learned. A search online will give you a whole bunch of free online statistical significance calculators to help you measure your results.

Once you’re satisfied that your test results can be relied upon, go ahead and roll them out, but continue to closely monitor performance and always be looking for opportunities to refine and optimize. 

Step 3 – Record your results

Forgive me if this one seems obvious, but I bet you don’t record all of your email testing results. Do you record all of the tests you did that proved to be ineffective?

Recording and documenting failure is one of the most overlooked aspects of marketing testing. I often find that inconclusive test results only exist in one or two of the minds within a marketing team. Very, very rarely do the inconclusive results find their way to a spreadsheet, document, or presentation. However, inconclusive results hold more value to your business than you may think. 

Firstly, recording results that aren’t particularly positive means that you are much less likely to repeat the same test again (unless you deliberately want to re-test it).  Secondly, it feels like you are getting somewhere by having a track record of all tests. You can see what you’ve tried, and if there’s a pattern of inconclusive tests, then you can pivot your testing strategy.

One final thing to consider is how valuable the records of email marketing test results are for other people to see. Recording your results will show your CMO and CEO, or anyone else for that matter, that you are milking your testing strategy for every possible opportunity to improve your email marketing. They would much rather see everything you tested with a management summary of findings, rather than just a verbal run-through of a couple of things you found that worked. What about new hires? A comprehensive record of everything you have tested would be really useful to them. What about copywriters in other teams or departments, designers, your agencies? All of them will find extreme value in a properly documented plan.

So there you have it – three steps to getting the most from email marketing split testing. Want to know more? Just ask away in the Comments section and I’ll be sure to get back to you. 


Philip Storey

About the Author: Philip Storey

Philip Storey is Global Head of Strategic Services at Lyris and has been specializing in email for a decade, working with brands to create engaging and response-triggering email for acquisition and retention strategies that drive high return on investment. He is also a trainer, speaker, and writer for several industry blogs such as Econsultancy and the IAB.

Leave A Comment