November 13, 2012

Why Attribution Efforts Fail Miserably: A Basketball Example

I want to walk you through a series of images from a basketball game.  The images will explain why attribution efforts fail miserably.

Look at the first image (below).  The player in the middle of the court just secured the rebound of a missed shot (the number on his jersey is "4").  He is passing the ball to the point guard (the number on his jersey is "0").


Now, we move a few seconds ahead in the action.  I positioned two arrows on the two players we are following.  The player with the ball (number "0") dribbled to the top of the key.  Next, he will pass the ball to the right wing, where his teammate is wide open.  But more important, look at the left arrow.  This is player number "4".  He has hustled down court, passing all of his teammates.  Because of his hustle, he had drawn two defenders near him.  And because he drew two defenders near him, his teammate (number "12") is wide open on the right wing.
Ok, we'll look at one more image, at the end of the play.
Because player #4 drew a defender close to him, his teammate on the right wing (#12) was wide open, and received a pass from player #0.  A defender rushed to the teammate (#12), blowing right past a shot fake.  Now, the teammate (#12) has a wide open three point jump shot attempt, which he buried!  The crowd went crazy, because this player hit a wide open shot that counted for three points, he's credited with one more point than he would be credited for on easier shots taken inside the three point arc.

But look at our player, player #4 ... he is just above the black arrow on the image.  If the shot is missed, player #4 has hustled into position, and has blocked out the defender.  Most long shots rebound to the opposite side the ball is shot from.  In other words, player #4 is in good position to get the rebound if the shot is missed ... and if he doesn't get the rebound, he has a defender blocked out, so that his teammate (to his left, #35) will get the rebound.

In the box score (basketball's version of a KPI attribution dashboard), here is what is documented from this offensive series:
  • Player #0 = 1 assist.
  • Player #12 = 3 points.
  • Player #4 = No credit for anything.
This is exactly the same situation we observe in all of our attribution/matchback work.  We'd give player #12 credit for closing the deal (last touch attribution).  We'd give player #0 credit for an assist (multi-touch attribution).  We'd give player #4 no credit for anything other than a defensive rebound.

Player #4, who is truly the reason for the success of this offensive series, gets no credit.  In large part, he gets no credit because we don't have a measurement system to assign value to his efforts (hustle, positioning, basketball IQ, fundamentals).

Coaches, however, do assign value to his efforts.  They watch film, and can visually see his value to the team.  That's why he is playing in this game.

We, as a measurement community, will fail until we find a way to provide attribution to the activities that truly make a purchase happen (customer service, merchandise, creative).  Today, we give too much credit to channels, discounts, and promotions (i.e. player #12 and, to a lesser extent, player #4).  

We don't give credit to customer service, merchandise, and creative (i.e. player #4).

6 comments:

  1. Anonymous7:27 PM

    I've been reading your blog for a few weeks, it's awesome. I wish I had found it before!

    ReplyDelete
  2. I would translate #4 into something similar as a View Through in Display. View through can be accounted for and put into an attribution model. That way #4 can get credit. In fact, if you (the coach) want him to get more credit that can be done as well.

    Yes attribution models struggle in some aspects but I don't think you should toss out the idea. I see the problem above as they are not properly tracking and assigning credit...they just need to fix it instead of just using whats there.

    ReplyDelete
  3. Nobody would disagree with you, Ryan. Makes perfect sense.

    And if you keep adding things into your attribution model, you're still not accounting for the things that matter most ... customer service, merchandise, creative ... and the organic percentage. I've yet to see an attribution expert choose to not take credit for orders that would happen anyway. Example - customer was going to purchase a pair of jeans from L.L. Bean - customer wants to find a free shipping code, goes online and finds one - attribution expert gives credit for the order to the affiliate that had the free shipping code. The darn jeans were what caused the order, and yet, the attribution expert gives no credit to the jeans! This is the essence of merchandise analysis and measuring the organic percentage.

    This is what I am getting at with this post ... player #4 is more like the jeans than anything else.

    ReplyDelete
  4. I agree that there are some inherit traits to the conversion past the marketing tactics.

    So how would you measure back revenue in a fair way that accounts for Organic percentages?

    ReplyDelete
  5. Many direct marketers do the following:

    (1) Catalog mail/holdout tests.
    (2) Email mail/holdout tests.
    (3) Display mail/holdout tests.
    (4) Search budget variations.
    (5) Matched market tests for offline advertising.

    The combination of tests allows the marketer to get a very good idea of the percentage of sales that happen absent of marketing (usually around 50% of the total ... your mileage will vary).

    In the example I just gave, you have to back 50% of all attributed sales out, and then run your attribution algorithm ... assuming you use the results of your mail/holdout and matched market tests to determine the attribution rules.

    ReplyDelete
  6. Totally agree with you Kevin.

    The start point should always be the customer profile: is the customer a new or returning customer?

    If they're a returning customer, did they purchase in their "purchase window" or not?
    If yes, then the source of traffic is probably less relevant than the product itself.
    If not, then did the traffic source or merchandising "reactivate" them? I'd side with the merchandise especially if the traffic source is branded/direct/email/etc.

    But if it's a new customer, then the acquisition traffic attribution analysis is what should be looked into more closely.

    My view is that what everyone in the industry calls marketing should be much more clearly delineated into acquisition and retention.

    The goal of traffic and marketing is to get customers to the site and initiate a product view or search. It's the site and product that complete the sale.

    ReplyDelete