Williams Sonoma always does a nice job of sharing fun facts with the public. In their third quarter earnings release, they state that "55% of online revenues are generated by customers who recently received a catalog."
This is always an interesting topic of debate in the database marketing world. Williams Sonoma does not specifically state which of two popular analytical methods they use to measure this metric.
Most popular, and most vigorously argued against by the analytically adept, is the method of attributing every online order to the catalog channel, if a customer recently received a catalog. The theory behind this technique (often called a "matchback analysis") is that the catalog inspired the order. Many vendors promote this methodology, and for good reason. The technique can overstate orders attributed to mailed catalogs, and vendors have a vested interest in promoting paper as a viable means of profitable marketing. Critics will argue that if you mail your entire housefile, this methodology will cause you to attribute every single online order to the mailing of the catalog. Critics will also argue that if you mail every housefile name a catalog, and send every housefile name an e-mail, the methodology completely breaks-down, rendering the analysis useless.
Less popular is the method of an "A/B" split. The marketer randomly splits her mail list into two halves. 50,000 customers receive the catalog, a like group of 50,000 customers do not receive the catalog. Several weeks after the in-home date, the marketer measures total sales in the mail group and control group, in both catalog and online (and, where applicable, retail) channels. This method tends to provide much less-optimistic answers than the "matchback analysis". Critics will argue that this methodology cannot produce reliable results due to sampling error issues.
Which methodology do you believe is more appropriate for allocating online orders to the marketing channel that drove the order?
Interesting post Kevin.
ReplyDeleteI'd personally be inclined to carry out the analysis by running the A/B test as you mention; though probably taking a much smaller control group of customers who don't receive a catalogue to mitigate potential loss of revenue in case the catalogue really is that effective.
In the analysis of attributing all catalogue recipients vs online orders I understand the matchback approach is more commonly used - albeit (IMHO) not a valid approach for the reasons you outline.
If emails are also sent out, then simple tagging and tracking via the web analytics tool can correctly attribute the last causing effect for the sale. Through we then want to start considering the complexity of the events that led to the sale - a common problem with web analytics (easy to attribute a search sale or a banner sale but how do different channels contribute to the sale - the real multi channel issue!). It's for this reason that the banner 'view-through' metrics are becoming less valuable in a multi channel world.
Not easy to answer without understanding who are catalogue recipients..
Best, James.
Thanks, Kevin, for throwing additional light on this topic.
ReplyDeleteIt is my experience that the percentage of online orders utilizing a catalog has sloped downward from a high of 95%+ some years ago to about 82% in 2004, around 78% in 2005 and, likely, 72-70% in 2006.
It seems clear, and irrefutable, that there is a growing shift from the paper catalog to search, and that this logical shift will continue.
One simple method for measurement which I first suggested in the keynote speech for the 1998 MeritDirect Co-Op conference was to add one required field to the online order/shopping cart (making sure the the question has to be answered before the order completes). That question is:
Did you use or refer to our paper catalog at any point for this order?
It may not be 100% accurate, but it will give you a reasonable read on this question fairly quickly. Regardless, it is knowledge that all catalog/online marketers--B2B, B2C or hybrid--must have in order to make informed decisions relative to channel mix budget allocations.
Best regards
Don Libey
www.libey.com