- Question for Email Marketers: Do you measure the success of email marketing by executing holdout groups, not mailing customers email campaigns for up to three months at a time?
Ask 100 email marketers this question, and fewer than five will answer "yes".
What a shame.
You see, all of the ways that email marketing interacts with the rest of your business are illustrated by email holdout tests.
Take a look at our example:
Here's the deal. Email marketers typically measure performance via opens/clicks/conversions, adding average order size to the strategy to yield demand per email delivered. In our case, over the course of a three month period of time. we get $4.94 demand per customer ... across 26 email campaigns (2x per week by 13 weeks) ... each campaign is generating $0.19 for a total of $4.94.
We run a profit and loss statement.
|Less Email Cost||$0.08|
That's some sweet action!
Except, of course, that this analysis is providing us with the wrong answer.
When we do not send email campaigns to customers, as measured via mail/holdout tests, we generate just $3.33 demand per customer, not the $4.94 as measured via opens/clicks/conversions. We know this, because in our example, one set of customers did not receive a single email campaign for three months, and continued to generate incremental demand.
Here's what the profit and loss statement looks like, with our new and more accurate style of measurement.
|Less Email Cost||$0.08|
Now, there's nothing wrong with $1.20 profit per customer ... but it is a lot less than $1.81 per customer, right?
Let's take a look at what happens to other channels, when email marketing is discontinued for ninety days.
Paid and Natural Search results decrease, by 11% and 16% respectively. In other words, email marketing causes searches to happen. In theory, you have to take 11% of your paid search marketing cost, and allocate it back to your email marketing budget. Yes, you have to do this, this is what attribution is all about.
Now, that doesn't make a difference in this case ... but when your email marketing program lacks a lot of productivity, well, it is enough to push the whole program under water.
Look at all other online marketing demand. When you don't mail email campaigns to a customer, that customer changes behavior. The customer re-directs demand away from email marketing, back to the website ... spending $6.57 per customer online instead of $4.35 per customer online. In other words, customers are using email as a navigational tool to get to the website.
Look at affiliate marketing demand. When emails are stopped, customers spend $0.24 each at affiliates ... but spend $0.34 each at affiliates when emails are delivered. Now, the affiliate is probably getting a cut of each order, right? Well, you have to attribute that cut on the $0.10 incremental difference, attributing that to email marketing, not to affiliates, because email marketing caused the affiliate order to happen.
Display / Retargeting are not significantly impacted, are they?
Look at social media ... 28% of social media demand is caused by email marketing. In other words, if you take email marketing away, your brand advocates don't have as much to yap about, and consequently, they spend a little bit less. In this example, email marketing causes social media demand to happen, so you want to know that, don't you?
Email has minimal impact, when it comes to tablets/mobile.
We learned that email marketing isn't as effective as we believe it to be, when measured via traditional channels. That being said, it's still highly profitable.
Tomorrow, we look at each channel, decomposing the role of each channel with/without advertising.