Angela Southall
Angela Southall 17 October 2017

Smash your customer acquisition targets with these AB tests

Don't second guess what will work best to drive new customer acquisition via referral. Plan a program of AB tests to maximise results and you could benefit from a 5-7x improvement in results!

There are few areas in life where you can start something new and expect it to be perfect right away. Referral marketing is no different. Rather than having had instant good luck with their referral programmes the most successful companies we work with have engineered their great results — through testing and optimisation.

As a marketing professional, you will no doubt be used to AB testing; where you take a group of people and divide them into two (or more) groups to test any aspect of a marketing campaign. Your control group will experience Version A and the test groups will see a variation of that.

In the referral marketing arena, this enables you to test any aspect of your refer-a-friend programme. Discount type, discount amount and creative being just a few of the elements to test.

Although that sounds quite straightforward, the reality is that in referral marketing it can be more complicated that you expect. The time between a friend recommending a brand and their friend making a purchase can sometimes be quite long. But to be a success, and to ensure a great customer experience, you’ll need to ensure the friend receives the reward they were promised, even if that is months later.

Here we take a look at the main things to ensure AB testing delivers referral success for your brand.

3 Vital Ingredients your Referral AB Testing Platform Needs

AB testing - your bread and butter for success

1) Tests run simultaneously

For true AB testing, the campaign for the Control and the Test segments must take place at the same time. This ensures that external factors, such as weather, economics or seasonal factors do not distort the results.

The risk of running one referral offer and then testing a different variant at a later date is that the groups may be unrepresentative of your normal traffic and skew your results.

2) Testing is by cohort

AB testing and cohort analysis are usually described as separate things. Cohort analysis is usually defined as studying the data on two separate groups over time.

AB testing by cohort brings together the benefits of both strategies. Each cohort receives a different test but within each cohort everyone has a consistent experience in terms of incentives and creative, even over long buying cycles.

Why is this so important for marketers?

No doubt, you’ll want to test different referral offers and messaging all the time. There might be seasonal events which you’d like to incorporate in your referral marketing (extra rewards around Black Friday or Christmas for instance), or you might have other business pressures which mean you have to substitute a different value reward.

Amending marketing offers is fine for new cohorts of customers. But not for customers you’ve already promised the incentive.

Imagine your favourite brand offering you £40 if you introduce a friend to the company and £40 for your friend. You love the brand and you know your friend is in the market for their product so you’re more than happy to refer them. Imagine if your friend goes on to buy from them 2 months later, but they only give you both £10. Not a great experience for the newly introduced customer. Even worse, you feel embarrassed for introducing them and your loyalty to the brand disappears like a puff of smoke.

AB testing by cohort means you can test away to your marketing heart's content, but the offer will always remain the same for each cohort.

3) Real time statistics to identify when to roll out winning tests

One of the biggest mistakes of AB testing is that the tests are stopped too early. This can lead to mistakes in rolling out “winning” tests.

To ensure that you make the right decisions you should run AB tests until you get a big enough sample and ensure that the results are statistically significant. The Mention Me referral platform automatically alerts you when a test reaches significance, but if you are not lucky enough to use our solution then there are many online tools to help your calculations, such as this one from Neil Patel.

Even if your tests reach significance quickly they should be run for a minimum of 2-3 weeks.

Why? You’ll know from experience that email open and click through rates vary significantly by day. Likewise, on social media there are optimal days and times to post.

If you are fortunate enough to get enough traffic to your referral programme in a few days you might reach significance, but the results will similarly be skewed by day of week.  

Interesting AB tests to try

Designing a test programme is where the real creative thinking power is put to the test.  Consider elements like the psychology of the brand’s customers, what will motivate them to share, the perceived value of a discount compared to a free gift or other types of rewards. Of course, you should also rigorously AB test the copy and design across devices.

  • Where to engage your customers

Your customers are most likely to refer your brand at the greatest point of delight. In most cases this is when they place or receive their order. It makes sense then to test how you display your referral offer at this point to maximise conversions. We have run this test many times, and it has always achieved the same result. A 2x improvement is achieved with the overlay option.

Promote at point of customer delight

  • Framing your offer

Another great test is how you frame the offer. Do you lead with the offer the friend will receive if they make a purchase (generous) or with the offer you’ll give the customer for the introduction (self-interested)?

Option 1: Generous Offer

Generous Offer - Give your friends something

Option 2: Self-interested offer

Self Interested Offer - get something for yourself

This appears to perform differently for different brands. Interestingly, in a recent OnePoll survey that we ran, when asked which offer they preferred, the overwhelming favourite with respondents was Option 2, with almost four times as many chosing the "self-interested" option over the "gererous" offer.

  • Reward Type

Testing  a % discount versus a £ discount is a popular test and can have a big impact on results.

Testing referral reward type

A recent test by one of our homewares clients saw a 61% increase in customer acquisition with a percentage discount compared to the “£ off” discount.

Our OnePoll research also reflected this result with % discount being the most popular offer, particularly for males, with 39% preferring this to a range of other offers. Females also liked this offer most (29%), but were also interested in free gifts and free delivery.

Depending on the demographics of your customer base then, you could well see a different result.

The upside of AB testing

Having run referral programmes for over 200 brands, we know that AB testing is essential. Without testing businesses are lucky to achieve 1-3% of new customer acquisition from the referral channel. With AB testing you are talking up to 10x that! We regularly see brands achieve 10-30% of new customers being acquired by referral with a well optimised programme.

Clearly then, AB testing can have a huge impact on the performance of your referral programme. If you can’t undertake AB testing by cohort you are leaving sales on the table. 

This article has previously appeared on the Mention Me Referral Marketing Blog.

Please login or register to add a comment.

Contribute Now!

Loving our articles? Do you have an insightful post that you want to shout about? Well, you've come to the right place! We are always looking for fresh Doughnuts to be a part of our community.

Popular Articles

See all
How to Review a Website — A Guide for Beginners

How to Review a Website — A Guide for Beginners

Whether you're a startup or an established business, the company website is an essential element of your digital marketing strategy. The most effective sites are continually nurtured and developed in line with...

Digital Doughnut Contributor
Digital Doughnut Contributor 7 January 2020
Read more
10 Factors that Influence Customer Buying Behaviour Online

10 Factors that Influence Customer Buying Behaviour Online

Now is an era where customers take the center stags influencing business strategies across industries. No business can afford to overlook factors that could either break the customer experience or even pose a risk of...

Edward Roesch
Edward Roesch 4 June 2018
Read more
McDonald's: the History and Evolution of a Famous Logo

McDonald's: the History and Evolution of a Famous Logo

McDonald's logo is one of the most recognizable in the world. What does the logo of this brand mean, how did it evolve and what is the secret to the success of McDonald’s fast food network?

Anna Kuznetsova
Anna Kuznetsova 24 October 2019
Read more
The 3 Most Important Stages In Your Presentation

The 3 Most Important Stages In Your Presentation

If you want to deliver a presentation on a particular topic and you have to prepare yourself for it you should make sure that you go through several very important stages in order to craft a compelling, persuasive and...

Nicky Nikolaev
Nicky Nikolaev 16 February 2016
Read more
7 reasons why social media marketing is important for your business

7 reasons why social media marketing is important for your business

Social media is quickly becoming one of the most important aspects of digital marketing, which provides incredible benefits that help reach millions of customers worldwide. And if you are not applying this profitable...

Sharron Nelson
Sharron Nelson 6 February 2018
Read more