September 02, 2007

Measuring Multichannel Results

Click on the image to enlarge it.

At some multichannel merchants, the E-Mail Marketers, Catalog / Direct Mail Marketers, Web Analytics Team, and Business Intelligence / SAS Programming team work together on projects.

These folks don't have to all reside in the same department.

But it's good for business when they all work together.

In this instance, the CMO wanted to send an e-mail marketing campaign on a Thursday afternoon, much to the consternation of the e-mail marketing team. See, the e-mail marketing team felt that campaigns should be sent on Monday or Tuesday, in order to maximize conversion rate. The Web Analytics team concurred.

The CMO is the CMO, however. She demanded a campaign be sent on Thursday. She wanted to drive sales in her retail stores, as well as online.

The e-mail team wanted to "test" the incremental value of this strategy. They set up a control group, a group that would not receive the e-mail campaign.

The e-mail marketing team was used to generating $0.25 of incremental sales online when sending a Tuesday e-mail campaign. The campaigns were seldom designed to drive retail traffic.

When the campaign was completed, the e-mail marketing team partnered with the web analytics team. The results weren't encouraging.

Open Rate = 20%.
Click-Through Rate = 30%.
Conversion Rate = 2%.
Average Order Size = $167.
Sales Per E-Mail = $0.20.

The e-mail marketing team shared the results with the business intelligence / SAS programming team. This group had access to retail sales and catalog/telephone transactions. They wrote a program to analyze sales by day. The results are outlined in the image at the top of this post.
  • When measured via e-mail marketing tools and the web analytics platform, it appeared that the e-mail campaign generated $0.20 sales per e-mail.
  • When results are measured by channel, by day, a different story reveals itself (of course, a good marketer will test for statistical significance, a step omitted here for brevity).
    • The e-mail campaign drove online sales on Thursday and Friday alone. Sales were not incrementally increased on Saturday, Sunday or Monday.
    • The e-mail campaign drove retail sales on Saturday and Sunday, high-traffic retail days. In total, an incremental $0.14 were generated at retail by the e-mail campaign.
    • The e-mail campaign appears to have cannibalized catalog/telephone sales.
    • In total, the e-mail campaign actually drove $0.29 sales across channels. The campaign was 45% more effective than e-mail analytics and web analytics tools suggested.
This example, similar to an actual campaign I recently analyzed, demonstrates the importance of having cross-functional teams work together in a collaborative manner.

Simple open rate, click-through rate and conversion rate metrics do not tell the entire story. Simple mail/control testing, analyzed off of an integrated database, can illustrate customer behavior that may not have been anticipated.

For direct mail and e-mail marketers, the findings highlight an important trend in multichannel marketing.
  • E-Mail and Direct Mail campaigns sent between Monday and Wednesday often benefit the phone and online channels.
  • E-Mail and Direct Mail campaigns sent on Thursday or Friday often benefit the retail channel.
Your job is to work together as a collaborative unit, illustrating the in-home date that is best for the customer, and is best for the profitability of your company. Your CMO can develop a more effective multichannel strategy if you give her the tools to do so!


  1. Anonymous7:44 AM

    Excellent article - we should be looking at every marketing campaign in this manner.

    One note of caution, when slicing and dicing your tests like this... make sure your samples are large enough to provide reliable inference. Test designs must account for the more granular reads.

  2. Sure ... and this can be very hard to do if you're a small business.


Note: Only a member of this blog may post a comment.

I Just Don't Understand It

E-commerce pros ... I'll address you tomorrow. Can I share something with the rest of you? I'm as busy right now as I've been in...