Article

RJ Talyor
RJ Talyor 22 May 2017

Marketing Experiments vs Tests: Why Marketers Need to Know The Difference

Both tests and experiments are critical for growth. But knowing the difference between these two approaches can help you advance your organization’s learning of what works to drive revenue. There’s a science to both approaches.

There is a stark difference between a marketing experiment and a test. In the past three months, I’ve talked to more than 100 digital marketers representing more than 75 leading brands about their approach to learning about new, new things. Across these conversations, where marketers tend to interchange the words “test” and “experiment,” I’ve discovered a surprising divergence of those who test and those who experiment with their marketing dollars.

Let me explain.

A marketing test typically includes the following elements:

  • A previous winner;
  • A challenger;
  • A set budget and timeframe;
  • An expectation that there will be a winner.

A marketing experiment includes the following elements:

  • A hypothesis;
  • Two or more challengers;
  • An unknown outcome;
  • A set budget, but an unknown timeframe;
  • An expectation of success, and a fear of failure.

Testers are typically found when the marketer is tied to a specific outcome and the mentality of, “we do experiments, as long as the experiments work.” Experimenters are found where there’s a company culture, directive, team, or budget that demonstrates the maxim “there are no failed experiments” mentality.

Both tests and experiments are critical for growth. But knowing the difference between these two approaches can help you advance your organization’s learning of what works to drive revenue. There’s a science to both approaches.

I recently discussed this on a podcast with chemist and professor Dr. Bill Scott, who heads the Distributed Drug Discovery (D3) project at IUPUI in Indianapolis, other universities in the United States, and globally. D3 seeks to educate undergraduate students in chemistry and biology while they apply their learned skills and understanding to helping solve a serious humanitarian challenge — discovering drugs for neglected diseases.

Dr. Scott’s initiative helps students truly innovate and experiment. A major part of a chemistry student’s curriculum consists of running tests with a known outcome, like creating a specific molecule, or testing a known method against a competing method. In the marketing world, think of this as an A/B Test — for example with subject lines: A subject line that previously generated “x” amount of opens and clicks is tested against a competing subject line.

Dr. Scott wanted to interject more organic experimentation in his students' curriculum, and he uses his D3 project to do so. Students in the program are encouraged to experiment by developing new molecules, used as research to develop drugs for underserved and underfunded diseases. Pretty awesome, right? Listen to the full podcast for more details.

As marketers, we struggle with similar challenges when it comes to experimentation: Where do we start? What new outcome do we need to aim for? What platforms should we use? How much money should we be investing? A true marketing experiment begins with a hypothesis backed by intelligent data. The hard parts are the expectations we set and the fear of failure. Without these experiments and ongoing innovation, our marketing remains at the risk of producing stagnant results. 

Please login or register to add a comment.

Contribute Now!

Loving our articles? Do you have an insightful post that you want to shout about? Well, you've come to the right place! We are always looking for fresh Doughnuts to be a part of our community.

Popular Articles

See all
Digital Marketing Vs. Traditional Marketing: Which One Is Better?

Digital Marketing Vs. Traditional Marketing: Which One Is Better?

What's the difference between digital marketing and traditional marketing, and why does it matter? The answers may surprise you.

Julie Cave
Julie Cave 14 July 2016
Read more
Top 10 B2B Platforms to Help your Business Grow Worldwide

Top 10 B2B Platforms to Help your Business Grow Worldwide

Although the trend of a Business to Business portal is not new but the evolution of technology has indeed changed the way they function. Additional digital trading features and branding has taken the place of traditional outreach methods to get in touch with targeted buyers or sellers.Here are some of the best and fastest growing global B2B platforms that are helping thousands of businesses in the world to grow and reach their international and local clients.

Salman Sharif
Salman Sharif 7 July 2017
Read more
Collection Of The Best Email Testing Tools Online

Collection Of The Best Email Testing Tools Online

Don’t be afraid of email testing. There are many free or freemium tools online that can help you with testing your SPAM score, deliverability and even the rendering of your email. We feature 30 email testing tools in this article. Check out the complete list!

Roland Pokornyik
Roland Pokornyik 31 October 2016
Read more
4 Important Digital Marketing Channels You Should Know About

4 Important Digital Marketing Channels You Should Know About

It goes without saying that a company can't do without digital marketing in today's world.

Digital Doughnut Contributor
Digital Doughnut Contributor 5 November 2014
Read more
The death of re-marketing and its repercussions on e-commerce

The death of re-marketing and its repercussions on e-commerce

Google has been taking steps to end re-marketing. A bold move, perhaps, and a worrying one for advertisers. Google has recently introduced an addition to their ad-blocking capability that allows consumers to block all your re-marketing ads.  Nor has the organisation stopped there: Google is still developing their ad blockers as you read this.

Niamh Reed
Niamh Reed 18 April 2018
Read more