Article

Roland Pokornyik
Roland Pokornyik 11 March 2016

5 Mistakes That can Ruin Your Email A/B Tests

Testing your templates, your copy, your timetable can be a great help, but most of the time it won’t give you the Holy Grail. You have to test major and minor changes also in order to narrow your email down to the perfect version.

Expecting instant and spectacular results

But if you change just something small in most cases the difference will not be so spectacular that you can tell it truly is the alteration that made a difference or some other factor.

It might take a while till you get statistically significant results, especially if you have a small email list. You may need to repeat your tests various times to be able to make a well established decision. You don’t have to be a pro in statistics, but an A/B split test significance calculator might be quite handy.

Not testing at the same time

If you have multiple versions of your email, you should test them at the exact same time. Except when you are testing for the ideal time to send your mail. In this case you should use the exact same template, copy, subject line and so on. The one and only difference should be in timing.

If you don’t test different versions simultaneously you risk getting inconsistent results. Recipients won’t react to your emails the same way at different times. It definitely matters if you send your emails in the morning or the afternoon, on the weekend or in the middle of the working week. Be really specific about the goals of your test, and thegeneral rule of thumb is to modify only a single element at the same time.

Running too many tests

It’s great news that you want to improve your email campaigns, and definitely A/B tests are a great way to do that, but you shouldn’t overuse testing. Too many tests mean overwhelming amount of data, which can be confusing and impossible to analyze in its entirety.

You can and should absolutely test for important changes, but keep the number of simultaneously running tests low so you can interpret the results properly.

Defining the wrong KPIs

Before you even think about starting a test, define it clearly what is the goal of your test, and how are you going to measure the results. A small difference between templates can increase your click through rate, but in the end you may end up with lower conversion (purchase, order, whatever) rate. You may be looking at decreasing open rates while missing the fact that simultaneously more people click on your links in the mail.

In most cases the final conversion won’t happen inside the email, so you have to be able to track your website properly to be able to make the right decisions.

Not testing your list right

If you make extreme changes to your email template, test with a smaller number of subscribers first – this way the unsubscribe rate will be less drastic if it goes wrong. This is also true for offers where you want to get the most conversions in a limited time: test only a (random) portion of your list (or segment) so you can refine your template and copy.

Obviously there are tons of other tips we should list here. But maybe a definitive guide to email A/B testing would make the most sense. What do you think? Would you like to read about the depths of A/B testing?

Please login or register to add a comment.

Contribute Now!

Loving our articles? Do you have an insightful post that you want to shout about? Well, you've come to the right place! We are always looking for fresh Doughnuts to be a part of our community.

Popular Articles

See all
Promote Your Blog On These 30 Places

Promote Your Blog On These 30 Places

Social Media channels are one of the best ways to promote your blog content, but you shouldn’t stop there. Besides Social Media, there are more available places on the web which can be a great marketing tool for your blog promotion. I’m bringing you 30 proven places where you can promote your blog content and get great results.

Aleksej Durdevic
Aleksej Durdevic 7 December 2016
Read more
Top 10 Digital Branding & Marketing Trends for 2017

Top 10 Digital Branding & Marketing Trends for 2017

It’s time to re-evaluate and rebalance the digital approach for your company. Here are the Top Digital Branding & Marketing Trends for 2017 to watch for. The probing minds at the Borenstein Group, a Top Washington DC Digital Marketing and Branding Agency, have done the homework for you. Use it or lose it.

Gal Borenstein
Gal Borenstein 7 December 2016
Read more
4 Important Digital Marketing Channels You Should Know About

4 Important Digital Marketing Channels You Should Know About

It goes without saying that a company can't do without digital marketing in today's world.

Digital Doughnut Contributor
Digital Doughnut Contributor 5 November 2014
Read more
What Mobile App Design Looks like in 2017

What Mobile App Design Looks like in 2017

They say ‘move with the time or the time will leave you behind’. Being a startup it is important for you that you understand the trends, and amalgamate them in your business in order to attain the targets.

Nasrullah Patel
Nasrullah Patel 6 December 2016
Read more
Digital Marketing Vs. Traditional Marketing: Which One Is Better?

Digital Marketing Vs. Traditional Marketing: Which One Is Better?

What's the difference between digital marketing and traditional marketing, and why does it matter? The answers may surprise you.

Julie Cave
Julie Cave 14 July 2016
Read more