Dear Catalog CEOs: Attribution Revisited

Dear Catalog CEOs:

Attribution, or "matchback" as it is sometimes called, is one of the most controversial topics in marketing.  It's the process of giving credit to marketing activities that theoretically caused an order to happen.

Last week, I asked measurement experts to give credit to the following sequence of activities, or at least provide logic into their thought process.  Here's the sequence of activities:

1. Customer receives a catalog on October 1.
2. Customer receives email campaign on October 2.
3. Customer receives email campaign on October 4.
4. Customer visits site on October 5 via paid search, branded term.
5. Customer visits your mobile website on an iPad on October 6 via affiliate website, purchases item featured in October 1 catalog, uses free shipping promo code from affiliate website.

I received numerous responses.  I want you to read each response.  Look at the diversity of thought.  Look at the inconsistency across experts.  Nearly every person has a different philosophy, a different point of view.  Some provide solutions.  Some ask questions.  No answer is right or wrong.  Many answers are in direct opposition of each other.

A few things I want for all of you to think about:

1. If you have traditional direct mail or catalogs, you already know the answer, because you execute mail/holdout tests, so you already know the impact of step one above.
2. If you are an email marketer, you already know the answer, because you execute mail/holdout tests, so you already know the impact of step two and step three above.
3. If you are a search marketer, you already know the answer, because you have extensively tested branded and non-branded spending levels, so you know the impact of your search budget on step four above.
4. If you've followed this blog for the past 6+ years, then you know of the importance of calculating the "organic percentage", the percentage of demand not impacted by marketing.  You already know that it is important to subtract all of the "organic" demand from the story above before starting the attribution discussion.

Many experts would look at the four points I just made, and laugh at me!

I include this post in the "Dear Catalog CEOs" series, because I want to elevate the discussion.  We will not solve the attribution problem with measurement-guru discussion.  We need Executive leadership.  Executives need to see the diversity of thought, and to provide a point of view.  Executives need to provide direction, and not simply trust the next vendor-based black-box solution.

With that in mind, here are the responses I received, unedited.  Discuss.

Kelly Lorenz:  Is the item only featured in the catalog? I ask because many marketers put out a catalog and then promote the items via email to extend the life of the promotion and ensure several touch points. 

My thought is that each touch point caused an impact - especially since it was over a short period of time. The three push efforts (catalog, 2 emails) caused the brand to be top of mind. The customer is then thinking about the items and does a search based on the brand because they remembered that the brand sells those things they are thinking about. 

After visiting the site on October 5, they probably did some more research to see if there was a better price or version of the item elsewhere, perhaps slept on it then decided to buy once they found the best possible price via an affiliate. 

The affiliate site should get the least attribution in my opinion based on this line of thinking, but it's really hard to draw this scenario out for every purchase! 

Peter O'Neill:  Did the customer open the catalog or either email? Did they even notice the name of the company? No data on either question. 

Were they prompted to search for the company due to the catalog or emails? Or was it due to wanting a replacement for a previous item? Or due to their neighbour recommending the brand? Again no data. Plus if they didn't log in, no visibility of the visit from that customer.

Any indication of the reason they accessed the affiliate website or why they chose to click through to this company? Any indication it was due to the catalog, emails, previous visit, affiliate offer or any other reason? Still no data on any of this.

Was it an impulse or considered purchase? If you asked the customer why they purchased that product from that company, would they be able to attribute this to one or more marketing efforts? If they were able to, would they have correctly interpreted their subconsious?

These missing data points suggest using technology to do true attribution is impossible. You can get many different answers but would any of them be accurate? More importantly, would any allow you to optimise your marketing spend in the future (am sure could improve but optimise)?

Instead, let me ask. What were the business objectives for each marketing campaign? How was success going to be measured against each business objective? What actions will you take based on performance against objectives? Did you run any hold out tests to assist in evaluating the impact of the catalog or emails? 

I think these questions are more important than trying to rely on incomplete data to guess what influenced a previous purchase. The answers will allow you to change your future marketing mix/spend to improve business performance & ROI.

Brian Clifton:  My thoughts are that you need to do some testing/segmentation on this - pulling one specific example simply leads to more questions and it may not be representative. For example:

-Compare this scenario with one where step 1 is missing (no catalogue)
-Compare this scenario with one where step 3 is missing (only 1 email sent)

What 5 is telling you is that there is a value-added benefit of using affiliates and that customers (this specific customer to be exact, hence why I suggest avoiding single scenarios) are price sensitive. You can then test different value offers with your affiliates.

Hope this helps, Brian

Jay Allen:  My first thought is, why are you sending iPad visitors to your mobile site? ;) 

In terms of attribution, I think the catalog has to be considered the driver assuming that the item wasn't featured in the emails. The emails likely reminded the consumer of your brand and that item. The affiliate site likely came from a brand search (if I had to guess). Affiliates are good at getting a price-sensitive customer to pull the trigger, especially if the customer feels like they are getting a special deal.

In terms of attribution, the key is being able to create a reasonable attribution scheme that can measure a million customer actions and drive intelligence. I don't think we're technologically there. It matters whether one customer follows the path above, or 10,000 customers follow it. 

If you have large numbers of customers looking on affiliate sites for free shipping offers, it may be time to simply offer free shipping. If you have a handful, the right strategy is to use the affiliates to give away free PAH to price-sensitive consumers and keep charging everyone else. 

Oh, and I'm betting this is Jennifer.

David P. Himes:  I attribute the order to the affiliate web site with free shipping promo.

Of course all of the other pieces are factors. However, a major part of direct marketing is overcoming inertia. There are lots of things I see advertised, that I say, "I'd like to have that".

So the issue is not just creating demand, but overcoming the inertia of not doing anything. It's easier for a customer not to do anything ... it takes less effort and costs less. 

So, in this example, it's the free shipping offer from the affiliate web site that overcame the inertia and generated an order. 

There is not evidence, from the data provided, that an order would have been placed in the absence of the free shipping offer.

Christian D.I.:  I would attribute 65% of deal to affiliate website promoting the free shipping promo code, it looks to me that was the triggering action (unless the customer was somehow aware of the coming promo, i.e. was interested in the deal since the beginning but he was reasonably sure a promo would arrive shortly). I give a 25% to first catalog campaign, but if the promo awareness proves correct the attribution are inverted. I give remaining 10% to email and paid campaigns, cannot be more precise without knowing customer's interaction and engagement level

Doug First: Good question Kevin!

I'm inclined to agree with David - attribute the order 100% to the affiliate, since it was the 'last touch' that triggered the order. 

Shopping behavior is an extremely complex and variable process. Modeling it (attributing demand) is always imperfect. You could easily have added other (possibly important) influencers (Pinterest, TV, Retail, etc.) to your scenario. 

But, the point of sale is really the moment of truth where the shopper has developed enough trust in the retailer/channel/offer to part with their hard earned cash.

Your implied question is ‘where do I best spend my ad money?’ Hold-out tests (as you’ve preached) are really the best way to find the incremental demand generated by each contact/media. Since testing is expensive – I’d start with the most costly elements in your marketing plan (probably catalog). 

Of course I'd also question the effectiveness of my affiliate offers if customers made up a significant portion of affiliate sales. 

Ryan Green:  FIRST thought is that this user is crossing devices. In most of the basic attribution models they only look at a cookie that is tied to a user’s browsing habits (one device). If you are using some of the advanced models then you can stitch the two cookies to one Visitor. In this instance you will have 2 different cookies for this ONE person…so be sure your tracking logic is in place! 

Next is that you will need to bring in some of the offline data such as the Catalog information. If these are for known customers you might be able to stich some of this data to the cookies…which is always great to tie online to offline! There are some Unified Customer Processes (“UCP”) that are great for this in attribution models. 

Next, is do you want to track View through credit? Meaning for the steps 2 & 3 in the string above…what if those users only open and don’t click? Do you still want to associate credit to that Email Open? The same applies to banners…do you want to see view throughs or only click throughs? They require some integrations but it is definitely doable! 

After you have your tracking in place to analyze the stream above…then you can get into the analysis. I like to look at things using Even or a Starter(30% credit)/Player(30% credit)/Closer(40% credit) weighted model. I think for this example you will need to create some specific logic in your attribution. For example if it is a sale of a computer…specify that the marketing touches that get credit should have attributes of computer advertising. It’s a lot easier said than done but there can be conditional logic you can build to achieve that level of granularity. That way you could ignore some marketing tactics that were for something completely irrelevant.