Smash your customer acquisition targets with these AB tests
Don't second guess what will work best to drive new customer acquisition via referral. Plan a program of AB tests to maximise results and you could benefit from a 5-7x improvement in results!
There are few areas in life where you can start something new and expect it to be perfect right away. Referral marketing is no different. Rather than having had instant good luck with their referral programmes the most successful companies we work with have engineered their great results — through testing and optimisation.
As a marketing professional, you will no doubt be used to AB testing; where you take a group of people and divide them into two (or more) groups to test any aspect of a marketing campaign. Your control group will experience Version A and the test groups will see a variation of that.
In the referral marketing arena, this enables you to test any aspect of your refer-a-friend programme. Discount type, discount amount and creative being just a few of the elements to test.
Although that sounds quite straightforward, the reality is that in referral marketing it can be more complicated that you expect. The time between a friend recommending a brand and their friend making a purchase can sometimes be quite long. But to be a success, and to ensure a great customer experience, you’ll need to ensure the friend receives the reward they were promised, even if that is months later.
Here we take a look at the main things to ensure AB testing delivers referral success for your brand.
3 Vital Ingredients your Referral AB Testing Platform Needs
1) Tests run simultaneously
For true AB testing, the campaign for the Control and the Test segments must take place at the same time. This ensures that external factors, such as weather, economics or seasonal factors do not distort the results.
The risk of running one referral offer and then testing a different variant at a later date is that the groups may be unrepresentative of your normal traffic and skew your results.
2) Testing is by cohort
AB testing and cohort analysis are usually described as separate things. Cohort analysis is usually defined as studying the data on two separate groups over time.
AB testing by cohort brings together the benefits of both strategies. Each cohort receives a different test but within each cohort everyone has a consistent experience in terms of incentives and creative, even over long buying cycles.
Why is this so important for marketers?
No doubt, you’ll want to test different referral offers and messaging all the time. There might be seasonal events which you’d like to incorporate in your referral marketing (extra rewards around Black Friday or Christmas for instance), or you might have other business pressures which mean you have to substitute a different value reward.
Amending marketing offers is fine for new cohorts of customers. But not for customers you’ve already promised the incentive.
Imagine your favourite brand offering you £40 if you introduce a friend to the company and £40 for your friend. You love the brand and you know your friend is in the market for their product so you’re more than happy to refer them. Imagine if your friend goes on to buy from them 2 months later, but they only give you both £10. Not a great experience for the newly introduced customer. Even worse, you feel embarrassed for introducing them and your loyalty to the brand disappears like a puff of smoke.
AB testing by cohort means you can test away to your marketing heart's content, but the offer will always remain the same for each cohort.
3) Real time statistics to identify when to roll out winning tests
One of the biggest mistakes of AB testing is that the tests are stopped too early. This can lead to mistakes in rolling out “winning” tests.
To ensure that you make the right decisions you should run AB tests until you get a big enough sample and ensure that the results are statistically significant. The Mention Me referral platform automatically alerts you when a test reaches significance, but if you are not lucky enough to use our solution then there are many online tools to help your calculations, such as this one from Neil Patel.
Even if your tests reach significance quickly they should be run for a minimum of 2-3 weeks.
Why? You’ll know from experience that email open and click through rates vary significantly by day. Likewise, on social media there are optimal days and times to post.
If you are fortunate enough to get enough traffic to your referral programme in a few days you might reach significance, but the results will similarly be skewed by day of week.
Interesting AB tests to try
Designing a test programme is where the real creative thinking power is put to the test. Consider elements like the psychology of the brand’s customers, what will motivate them to share, the perceived value of a discount compared to a free gift or other types of rewards. Of course, you should also rigorously AB test the copy and design across devices.
- Where to engage your customers
Your customers are most likely to refer your brand at the greatest point of delight. In most cases this is when they place or receive their order. It makes sense then to test how you display your referral offer at this point to maximise conversions. We have run this test many times, and it has always achieved the same result. A 2x improvement is achieved with the overlay option.
Another great test is how you frame the offer. Do you lead with the offer the friend will receive if they make a purchase (generous) or with the offer you’ll give the customer for the introduction (self-interested)?
Option 1: Generous Offer
Option 2: Self-interested offer
This appears to perform differently for different brands. Interestingly, in a recent OnePoll survey that we ran, when asked which offer they preferred, the overwhelming favourite with respondents was Option 2, with almost four times as many chosing the "self-interested" option over the "gererous" offer.
Testing a % discount versus a £ discount is a popular test and can have a big impact on results.
A recent test by one of our homewares clients saw a 61% increase in customer acquisition with a percentage discount compared to the “£ off” discount.
Our OnePoll research also reflected this result with % discount being the most popular offer, particularly for males, with 39% preferring this to a range of other offers. Females also liked this offer most (29%), but were also interested in free gifts and free delivery.
Depending on the demographics of your customer base then, you could well see a different result.
The upside of AB testing
Having run referral programmes for over 200 brands, we know that AB testing is essential. Without testing businesses are lucky to achieve 1-3% of new customer acquisition from the referral channel. With AB testing you are talking up to 10x that! We regularly see brands achieve 10-30% of new customers being acquired by referral with a well optimised programme.
Clearly then, AB testing can have a huge impact on the performance of your referral programme. If you can’t undertake AB testing by cohort you are leaving sales on the table.
This article has previously appeared on the Mention Me Referral Marketing Blog.