A Question For E-Mail Multichannel Marketers

Ok e-mail marketers, here's a question for you:


You send a catalog to customers on Monday, expecting to get $3.00 demand per catalog.

Next, you send an e-mail campaign to customers on Wednesday, expecting to yield $0.20 demand per e-mail sent.

A customer receives both the catalog, and the e-mail campaign. The customer does not open the e-mail campaign, does not click-through the e-mail to the website.

On Sunday, the customer visits the website, and chooses to order merchandise. The customer spends $100.00.


I have a feeling that the catalog marketers want credit for the full $100.00 order.

I want to hear from you, the e-mail marketer. How much of the $100.00 order should be allocated to the marketing activity you executed on Wednesday?

Matchback Analysis

Yesterday's post about allocating online orders to the catalog channel causes one to think about the best ways to measure advertising effectiveness.

Assuming you don't execute "A/B" tests to measure incremental sales across all channels, the matchback analysis provides better answers than doing nothing. However, there are challenges that need to be addressed.

Example #1: You send a catalog on October 15 to your entire catalog housefile. You send an e-mail campaign on October 18 to your opt-in housefile list. You also secure a placement on MSN on October 19. On October 22, a prior catalog purchaser and current e-mail subscriber visits your site with a referring URL from the MSN placement, and purchases merchandise. Which advertising channel is responsible for this order?

Example #2: Same criteria as above (catalog mailed on 10/15, e-mail sent on 10/18, MSN placement on 10/22). Customer visits site after conducting a search on MSN on 10/22, and purchases merchandise. Which advertising channel is responsible for this order?

Example #3: Same criteria as above. Customer visits your website via organic search on Google on 10/24, and purchases merchandise. Which channel is responsible for this order?

What analytical technique do you use to reconcile the issues in examples one through three? If you don't manage this issue in-house, which vendor do you work with to resolve these issues? If you don't work with a vendor or analyze these issues in-house, how do you determine your educated guesses?

Lastly, who are the vendors that you believe are most credible and most talented in this field of study?

Williams Sonoma: Incremental Online Sales and Matchback Analysis

Williams Sonoma always does a nice job of sharing fun facts with the public. In their third quarter earnings release, they state that "55% of online revenues are generated by customers who recently received a catalog."

This is always an interesting topic of debate in the database marketing world. Williams Sonoma does not specifically state which of two popular analytical methods they use to measure this metric.

Most popular, and most vigorously argued against by the analytically adept, is the method of attributing every online order to the catalog channel, if a customer recently received a catalog. The theory behind this technique (often called a "matchback analysis") is that the catalog inspired the order. Many vendors promote this methodology, and for good reason. The technique can overstate orders attributed to mailed catalogs, and vendors have a vested interest in promoting paper as a viable means of profitable marketing. Critics will argue that if you mail your entire housefile, this methodology will cause you to attribute every single online order to the mailing of the catalog. Critics will also argue that if you mail every housefile name a catalog, and send every housefile name an e-mail, the methodology completely breaks-down, rendering the analysis useless.

Less popular is the method of an "A/B" split. The marketer randomly splits her mail list into two halves. 50,000 customers receive the catalog, a like group of 50,000 customers do not receive the catalog. Several weeks after the in-home date, the marketer measures total sales in the mail group and control group, in both catalog and online (and, where applicable, retail) channels. This method tends to provide much less-optimistic answers than the "matchback analysis". Critics will argue that this methodology cannot produce reliable results due to sampling error issues.

Which methodology do you believe is more appropriate for allocating online orders to the marketing channel that drove the order?