Attribution Simulations: Vendors And Clients, A Product Opportunity For You

Ok, we've spent three weeks talking about Attribution Simulations. The benefits are obvious, there's no arguing that.

And your current attribution vendor is highly unlikely to incorporate attribution results into a five year forecast, allowing you to simulate how different marketing channels impact the future health of your business.

So, I'm going to make the latest version of Attribution Simulations available to my clients, and make it available to the vendor community.

Kevin's Clients:

• Cost = $29,000, regardless of business size. Contact me now (kevinh@minethatdata.com) to get your Attribution Simulation started.

Vendors - You Have Two Choices:

• Contact me for the price (I expect to sell about eight of these things a year, to give you an idea of my sales potential - imagine what your sales potential might be) ... I will teach you how I create the simulations, soup to nuts. Your job is to convert the Attribution Simulation into software that your clients can use. I get to keep creating Attribution Simulations for my current/future clients, you get to sell the tool to your current/future clients. Contact me (kevinh@minethatdata.com) for additional details. If you are a catalog co-op, or database provider, then this solution is right up your alley. Heck, the simulations typically demonstrate that catalogers under-invest in new customer acquisition - the simulation will help your cause more than it will help clients (well, it will help both, no doubt).
• Exclusive Use ... I will accept bids through January 15 ... if you submit the highest bid (contact me at kevinh@minethatdata.com to learn the minimum bid amount), you get exclusive use of the Attribution Simulation ... no other vendor will be allowed to purchase the educational process from me. But, again, bids will only be accepted through January 15.

I think we all agree that the combination of Attribution algorithms and a futuristic Simulation framework yields something that any CEO / CFO / CMO would love to use. Let's get busy using Attribution Simulations!

Attribution: Advertising Curves

When I run Attribution Simulations, I have to estimate how incremental ad dollars will impact future demand.

In other words, if I increase my email budget by 50%, how much will total demand attributed to email increase by? 5%? 25%? 50%?

Any Attribution Simulation that forecasts five year volume require a series of assumptions.

Look at the blue line in the graph. This curve assumes that each dollar spent yields a linear and consistent increase in attributed demand.

Look at the green line in the graph. This is the most common outcome - each incremental dollar spent yields less volume, in accordance with a law of diminishing returns.

The brown line in the graph is what we typically see in email marketing - one campaign a week gets you most of the benefit, with marginal benefit observed with increased campaigns each week.

I typically use power functions in my Attribution Simulations, though you are welcome to use any curve you like, as long as the curve represents reality.

The magic of any Attribution Simulation comes from the advertising curves built into the simulation. The advertising curves are the secret sauce, folks.

Attribution: The Interaction Matrix

In your typical attribution discussion, there is considerable consternation regarding first-click and last-click attribution ... and everything in-between!

Honestly, most of the talk is theory. Everything, and I mean everything in attribution is wrong. And that's ok.

Since the vast majority of my clients use a hybrid of matchback (first-click) and last-click attribution, I calibrate my simulations around a combination of both.

The "Interaction Matrix" is where I build rules. Since many of my clients are catalogers, folks who execute mail/holdout tests, I get access to some juicy information!

In the mail/holdout tests (for both catalogs and for email), I can identify what happens if marketing is discontinued. This allows me to estimate the importance of support channels, like paid search.

For instance, it's common to see paid search distribute something like this:

• 50% of all paid search volume disappears if catalogs are discontinued.
• 25% of all paid search volume disappears if emails are discontinued.
• 50% of all paid search volume is "dark matter" - unmeasured - but shows up in retail purchases.

I enter these factors into my "Interaction Matrix". This allows me to see what happens to the overall business if I ramp-up catalog marketing or email marketing, for instance.

Here, I'm going to increase my catalog budget by 50%, and increase my email marketing budget by 50%. Look at what happens to the paid search budget:

On a last-click basis, we see that the increase in the catalog budget and email budget resulted in more paid search last-click demand.

We can also see that the paid search budget increased significantly, without interaction on our behalf. In other words, by sending out more catalogs, and by sending out more email campaigns, we drive more customers to Google (which both Google and Amazon heartily endorse). Among the customers not lost to the dark matter of the e-commerce world, we generate more paid search demand, reflected in this simulation.

This is why I like focusing on Attribution Simulations. I get to see how all of these different "omnichannel" marketing strategies interact and play out over the next five years. I'm pretty confident you'd like to have a tool like this, at your disposal, as well.

Attribution: What Happens If You Terminate Your Email Marketing Program?

Too often, I sit in meetings where business leaders ask very simple questions of the marketing team.

And too often, the marketing team cannot answer simple questions.

Here's our business from yesterday. Let's pretend that the Executive Team is not "giving the love" to the email marketing team. Yup, happens all the time.

What we don't ever get to see is what happens when email marketing is stopped.

Yes, stopped.

I zero out the email marketing line, for each of the next five years, in my Attribution Simulation. Here's the outcome, and it isn't pretty, folks.

Oh. My. Goodness.

The Attribution Simulation shows us that email costs us +/- ten million dollars in the first year.

However, by not offering email marketing, we do not reactivate or activate as many customers. We cut back on new customers. The result is a long-term death spiral - the business is down to $58 million in year five, instead of $81 million.

In other words, email marketing has a $10 million dollar short-term impact, and a compounding impact over the next five years of up to $13 million additional volume.

And take a look at profitability - it's virtually gone from this business, isn't it?

Now, if you are the email marketing Director, wouldn't you like to have this Attribution Simulation in your back pocket? Heck, over time, you're responsible for almost all of the profit generated by this business.

This is the power of the Attribution Simulation.

Contact me (kevinh@minethatdata.com) for your own Attribution Simulation.

Attribution: Why Retailers Send A LOT Of Emails To Us

From time to time, I'm asked questions like "why do retailers insist on spamming us with daily email campaigns?"

There's two key reasons.

First, there is essentially no cost to email marketing. Oh, I know, you're going to howl at me about how you have staff and how you pay your vendor a fortune (and you do).

But once you bury those fixed costs, the cost to send one incremental campaign is as close to zero as anything in marketing not called social media.

Pretend you are CEO of the company featured above. Your business is forecast to drop, year after year after year. Your job is to fix this problem. Fixing the merchandise assortment might take 1-3 years. You don't have that kind of time.

So you hire Kevin, and Kevin runs an attribution simulation for you, forecasting what happens over the next five years if you choose to double your email contact frequency - from two per week to four per week. Here's the outcome:

Interesting, don't you think? The drop in business volume isn't so severe, anymore. Profit improves marginally. Heck, retail sales increase by 2% a year, and you barely had to do a thing make it happen ... do you know how hard it is to grow retail comp store sales by 2%? Hint - it's really, really hard!

When you have a low variable cost channel like email marketing, and you have to drive customers into retail stores, you're going to need to send an awful lot of email campaigns to customers.

When I run attribution simulations, I make sure I thoroughly understand how the business could grow with minimal expense and effort. Email is one of the channels that gains priority.

Tomorrow, I'll show you what happens to this business when email marketing is discontinued.

Attribution: The Future Matters

Here's a typical Attribution challenge.

Maybe you're spending a lot of money on Paid Search. Your Attribution vendor suggests you're overpaying, significantly overpaying for many keywords. So you cut back on spend. You've just "optimized" your business. Good for you!

And then, nine months later, your business is plowing through +2% gains to last year, when you expected +6% gains to last year. Management is upset. Nobody knows why business is not meeting expectations, all anybody knows is that something is just a little bit off.

Here's something that is not well understood about Attribution. When you optimize based on past results, you do not factor in future performance. As a result, future performance is not optimized.

Here's an example - you have a customer with a 40% chance of purchasing again next year. But you are significantly overspending in Paid Search - so you back off numerous terms. For this individual customer, maybe one who likes Paid Search, the repurchase rate changes ... it was 40%, maybe now it is 30%.

If the customer has a 30% chance of purchasing, instead of 40%, then the customer will obviously spend less - and will spend less across the channels the customer prefers (maybe affiliates, maybe email) - since paid search closes sales generated in other channels. In other words, email productivity may suffer because the customer is less active.

Well, it will suffer. I've run a lot of downstream simulations, and that's what I observe.

In other words, the optimization of short-term results via Attribution leads to the sub-optimization of the business in the long-term.

This is a problem that needs a solution. In upcoming posts, we'll talk about the solution.

Attribution

Attribution is terribly important these days, don't you think? From the inaccurate and simplistic matchbacks that revolutionized the catalog industry ten years ago to the multi-touch models that incorporate social/mobile activities, we're almost forced to employ some sort of Attribution model in our results analytics.

An Attribution model provides an estimate of the impact each advertising channel had on an order. If a customer purchased after touching catalog - email - email - social - affiliate, we don't want to give all credit to the catalog (matchbacks), or all credit to the affiliate (last touch). Both methods are wrong.

In fact, every Attribution methodology is wrong. And that's ok. We simply cannot crawl inside the head of a shopper in Upstate New York and divine exactly why a purchase happened. 

Everybody has to make a guess at some point, everybody has to infer just how effective each marketing channel is.

There is a piece of the Attribution puzzle that is missing. It's the forward-looking piece.

Attribution has a serious limitation. You look backwards, and you make a guess as to whether marketing activities were profitable or not. Not many folks "do the opposite" ... not many folks look forward, and estimate how the future of the business changes as you make changes to the advertising budget.

In upcoming weeks, we'll explore a "future" view of Attribution - combining the work of your favorite Attribution vendor with a five year estimate of demand/profit given different marketing investments.

Come along for the ride, ok?!

Attribution Beyond Catalogs

Yesterday, I outlined the methodology I use to perform catalog attribution (click here please).  The article got good numbers, and generated questions.  This is the theme of the most commonly asked question.

Question:  You only care about catalogs.  We live in an omnichannel world.  How do I account for paid search, you moron?

Paid Search is the most complicated case, simply because we have to also ascertain the click-through rate within catalog / email driven searches.

First, we need to have both catalog and email holdout test results available.  You have email holdout test results available, right?  Right?  Because if you're going to do attribution work, you're going to apply real science, not just hokum-based guesses used by other practitioners.  Promise me that you have catalog and email holdout results.  If not, don't go further, you're just guessing, and that's really dangerous.

Here are the results from a sample catalog holdout test.

And here are the results from a sample email holdout test.

We only need a few additional pieces of information to identify the profitability of paid search.

First, in the catalog test, we learn that 50% of paid search demand is catalog-driven.  In other words, if we took catalog marketing away, 50% of paid search demand disappears.  Therefore, half of paid search demand is immediately allocated to catalog marketing.

Second, in the email test, we learn that 10% of paid search demand is email-driven.  In other words, if we took email marketing away, 10% of paid search demand disappears.  Therefore, 10% of paid search demand is immediately allocated to email marketing.

Let's say that our paid search program possesses the following metrics.

• Total Budget = $1,000,000.
• Total Clicks = 2,000,000.
• Conversion Rate = 2%.
• Average Order Value = $100.
• Total Demand = $4,000,000.
• Flow-Through Rate to Profit = 40%.
• Total Profit = $4,000,000 * 0.40 - $1,000,000 = $600,000.

Here's where things get a little bit messy.  You need to know the conversion rate of clicks attributed to catalog marketing, and to email marketing.  Few people possess this knowledge.  Go talk to your analytical gurus, vendors, or Google Analytics ninjas, and have them derive this number for you.

Let's pretend we know this number.

• Catalogs and Email Paid Search Conversion Rate = 2.5%.

Ok, we're making progress now.  Let's calculate the conversion rate for non-catalog and non-email clicks.  First, we know that 60% of paid search demand is caused by catalog and email marketing.  So we subtract that out of the equation.

• $4,000,000 * (1 - 0.60) = $1,600,000.

In other words, $2,400,000 paid search demand is catalog and email driven.  We know that the average order value is $100, we know that the conversion rate is 2.5%.  Therefore, we can calculate catalog/email driven clicks:

• $2,400,000 demand / $100 AOV = 24,000 orders.
• 24,000 orders / 0.025 = 960,000 clicks.

If 960,000 clicks are catalog/email driven, then 1,040,000 have to be paid search driven.

• 2,000,000 - 960,000 = 1,040,000.

Let's run the profit and loss statement for catalog/email driven clicks.

• Total Clicks = 960,000.
• Total Budget = 960,000 * $0.50 = $480,000.
• Conversion Rate = 2.5%.
• Average Order Value = $100.
• Total Demand = $2,400,000.
• Flow-Through Rate to Profit = 40%.
• Total Profit = $2,400,000 * 0.40 - $480,000 = $480,000.

By simple subtraction, we can calculate the impact of paid search, outside of catalog/email.

• Total Clicks = 1,040,000.
• Total Budget = 1,040,000 * $0.50 = $520,000.
• Conversion Rate = (1,600,000 / $100) / 1,040,000 = 1.54%.
• Average Order Value = $100.
• Total Demand = $1,600,000.
• Flow-Through Rate to Profit = 40%.
• Total Profit = $1,600,000 * 0.40 - $520,000 = $120,000.

There you have it.  You just attributed paid search driven orders to catalog and email marketing, and you know what remains.  What remains is still profitable, though it converts at a much lower rate.

Now, you have yourself a dilly of a pickle here.  It's not terribly easy to identify a customer as a catalog/email driven visitor to Google, then make separate decisions based on that information.

Because of that, some of the attribution talk is nonsense.

Let's pretend that the non-catalog and non-email clicks were unprofitable.  You have to have a working relationship with Google that allows you to tell Google, at the time somebody visits Google, that the visitor is catalog or email driven - and if not catalog/email driven, don't pay for the click.

So if you can do that, then the attribution exercise is actionable.

If not, then the attribution exercise is done for knowledge, but is not actionable.

This process, of course, is repeated for all advertising channels.  Demand/Expense are allocated to catalogs and email marketing, with the remainder allocated to each individual marketing channel.

If the remaining marketing channels are shared, most would just allocate fractionally, based on pre-determined rules.  This, of course, is largely hokum, but there is a market where people are willing to pay for hokum-based research, so be it.

ACTION REQUIRED FOR CATALOG PROFESSIONALS: NAME YOUR FAVORITE MATCHBACK VENDOR

This is a picture of the Red Cross.  They are in the process of responding to a disaster.

If you had to attribute events that caused the Red Cross to be here, you'd have a challenge on your hands, right?  It would be easy to demonstrate that the disaster gets most of the credit.

It would be hard to show how individual contributions and corporate contributions led to this response.  It would be hard to show how the actions of a Red Cross volunteer or employee led to the purchase of the vehicles, which contributed to the response.

Similarly, in catalog marketing, we have to attribute response.  In the rest of the marketing world, this is called "attribution".  In catalog marketing, catalogers frequently take credit for actions that should be attributed to online marketing ... as a result, the methodology is called "matchback", not "attribution".

There are many matchback vendors.  Often, I'm asked to comment on the strengths and weaknesses of matchback vendors.

Instead of my opinions, let's focus on your thoughts.

In the comments section, please leave an anonymous comment.  Here's what I want you to share.

1. Name of the Matchback vendor you use.
2. Grade you would give their services ... "A", "B", "C", "D", or "F".
3. Tell us what your matchback vendor does best.
4. Any additional considerations.

If you wish to put your name behind the comments, go ahead.  If not, please leave an anonymous comment, and give your thoughts to the catalog community.  Via the magic of search engine optimization, this can become the post that folks look to for your thoughts about matchback vendors.

Attribute This!

Maybe you watched the Kennedy Center Honors event earlier this week.  If you did, you witnessed this performance of Led Zeppelin's "Stairway To Heaven" ... performed by Ann and Nancy Wilson.

You had a band, an orchestra, a choir, and Ann Wilson belting out vocals.  By most accounts, this was a spectacular performance, if you like music from this genre.

If you work in my industry, you'd be asked to parse this performance, attributing the reasons why it was successful.  Based on the elements that were successful, you'd be asked to invest more in the areas that yielded success, less in other areas.

1. What percentage of success do you attribute to Ann and Nancy Wilson?
2. What percentage of success do you attribute to the living members of Led Zeppelin being in attendance, being honored, enjoying a version of a song they created, what, 35 years ago?
3. What percentage of success do you attribute to the choir?
4. What percentage of success do you attribute to the orchestra?
5. What percentage of success do you attribute to the band?
6. What percentage of success do you attribute to the fact that the drummer passed away years ago, leaving his son to play drums in this performance?
7. What percentage of success do you attribute to Jimmy Page, at the 3:58 mark of the video, checking to see that the guitar solo is played properly?  Or to the soul playing the guitar at the 3:58 mark of the video?
8. What percentage of success do you attribute to the lighting?
9. What, specifically, caused Robert Plant to have tears in his eyes?

It's funny.  When it comes to content / product / merchandise, we'd never think of tearing the thing into bite (or byte) sized bits, would we?  The whole is greater than the sum of the parts.  We fully accept that you don't separate the choir from the orchestra and assign value to each.

In marketing, we want to reverse engineer everything, parsing the parts so we can sum things up on a scorecard, regardless how accurate or inaccurate we are.

Just a little something to think about as we head into the New Year.

Why Attribution Efforts Fail Miserably: A Basketball Example

I want to walk you through a series of images from a basketball game.  The images will explain why attribution efforts fail miserably.

Look at the first image (below).  The player in the middle of the court just secured the rebound of a missed shot (the number on his jersey is "4").  He is passing the ball to the point guard (the number on his jersey is "0").

Now, we move a few seconds ahead in the action.  I positioned two arrows on the two players we are following.  The player with the ball (number "0") dribbled to the top of the key.  Next, he will pass the ball to the right wing, where his teammate is wide open.  But more important, look at the left arrow.  This is player number "4".  He has hustled down court, passing all of his teammates.  Because of his hustle, he had drawn two defenders near him.  And because he drew two defenders near him, his teammate (number "12") is wide open on the right wing.

Ok, we'll look at one more image, at the end of the play.

Because player #4 drew a defender close to him, his teammate on the right wing (#12) was wide open, and received a pass from player #0.  A defender rushed to the teammate (#12), blowing right past a shot fake.  Now, the teammate (#12) has a wide open three point jump shot attempt, which he buried!  The crowd went crazy, because this player hit a wide open shot that counted for three points, he's credited with one more point than he would be credited for on easier shots taken inside the three point arc.

But look at our player, player #4 ... he is just above the black arrow on the image.  If the shot is missed, player #4 has hustled into position, and has blocked out the defender.  Most long shots rebound to the opposite side the ball is shot from.  In other words, player #4 is in good position to get the rebound if the shot is missed ... and if he doesn't get the rebound, he has a defender blocked out, so that his teammate (to his left, #35) will get the rebound.

In the box score (basketball's version of a KPI attribution dashboard), here is what is documented from this offensive series:

• Player #0 = 1 assist.
• Player #12 = 3 points.
• Player #4 = No credit for anything.

This is exactly the same situation we observe in all of our attribution/matchback work.  We'd give player #12 credit for closing the deal (last touch attribution).  We'd give player #0 credit for an assist (multi-touch attribution).  We'd give player #4 no credit for anything other than a defensive rebound.

Player #4, who is truly the reason for the success of this offensive series, gets no credit.  In large part, he gets no credit because we don't have a measurement system to assign value to his efforts (hustle, positioning, basketball IQ, fundamentals).

Coaches, however, do assign value to his efforts.  They watch film, and can visually see his value to the team.  That's why he is playing in this game.

We, as a measurement community, will fail until we find a way to provide attribution to the activities that truly make a purchase happen (customer service, merchandise, creative).  Today, we give too much credit to channels, discounts, and promotions (i.e. player #12 and, to a lesser extent, player #4).  

We don't give credit to customer service, merchandise, and creative (i.e. player #4).

Your Opinion Wanted: Attribution

Please use the comments section to offer your opinion as to how to attribute this order properly --- I am not judging whether any answer is right or wrong --- I am simply seeking opinions:

1. Customer receives a catalog on October 1.
2. Customer receives email campaign on October 2.
3. Customer receives email campaign on October 4.
4. Customer visits site on October 5 via paid search, branded term.
5. Customer visits your mobile website on an iPad on October 6 via affiliate website, purchases item featured in October 1 catalog, uses free shipping promo code from affiliate website.

Describe the thought process you go through to make your attribution guess.  This is really what I am looking for --- your thought process.  So please provide that to our readers in the comments section, thanks.

Attribution Week: Your Website and Paid Search

Take a good, hard look at the "All Other Online Demand" row in this table.


Which column has the best performance for the online channel?


The website is pulled in two opposite directions.

1. Catalogs cause website demand to happen.
2. Emails significantly cannibalize website demand.

Pure website demand is the most elastic of any channel.  Some forms of marketing drive a customer to a website, while other forms of marketing drive the customer away from the website.

It is easy to understand the impact of Catalogs and Email Marketing on website demand.  All we have to do is execute mail/holdout tests.

In Paid Search, it's important to geo-target.  You see, we want to measure the impact that Paid Search has on all other channels.  To this point, we've described the impact that Catalogs and Email Marketing have on Paid Search.

You can set up a brief test ... geo-target specific areas, then execute normal Paid Search activities vs. minimal Paid Search spend, and overlay this with Email mail/holdout test results.

This will clearly tell you how important Paid Search is to all other channels.

Ok, given what you've learned this week (and it is a lot), answer these questions.

1. Do you currently have this level of business intelligence at your company?
2. If the answer is "no", are you willing to execute the tests necessary to answer these questions?
3. If the answer to both questions is "no", please describe in the comments section why you don't think you need to answer these questions?  Go!!

Attribution Week: Channel Impact

This is our table from earlier in the week, measured via a 90 day long mail/holdout test of catalog marketing and email marketing.


There are many nuggets of information in the table.  Let's summarize how each channel is impacted by catalog marketing and email marketing.


Paid Search:  We generate $2.00 per customer via Paid Search in the quarter.

• Catalog causes $0.72, or 36% of Paid Search volume to happen.
• Email causes $0.18, or 9% of Paid Search volume to happen.
• 55% of Paid Search demand is independent of Catalog or Email marketing.
• We have to attribute 36% of Paid Search volume/cost back to catalog marketing, and we have to attribute 9% of Paid Search volume/cost back to email marketing.

Natural Search:  We generate $2.00 per customer via Natural Search.

• Catalog causes $0.58, or 29% of Natural Search volume to happen.
• Email causes $0.18, or 9% of Natural Search volume to happen.
• 62% of Natural Search demand is independent of Catalog or Email marketing.

Affiliates:  We generate $0.50 per customer via Affiliates.

• Catalog causes $0.34, or 68% of Affiliate demand to happen.
• Email causes $0.10, or 20% of Affiliate demand to happen.
• 12% of Affiliate demand is independent of Catalog or Email marketing.
• We have to attribute 88% of Affiliate marketing costs back to catalogs or email campaigns.
• Since Affiliate marketing is directly tied to Catalogs/Email, we need to strongly consider if the demand will continue to happen if Affiliates are dropped?

Display:  We generate $0.25 per customer via Display:

• Catalog causes $0.15 per customer of Display demand to happen (60%).
• Email causes $0.01 per customer of Display demand to happen (4%).
• 36% of Display demand is not driven by other marketing activities (36%).
• We have to attribute 60% of Display costs back to the catalogs that caused the demand to happen.

Social Media:  We generate $0.15 per customer via Social Media.

• Catalog causes $0.01 per customer of Social Media demand to happen (7%).
• Email causes $0.04 per customer of Social Media demand to happen (27%).
• 66% of Social Media demand happens independent of Catalog/Email.
• Given the strong link between Email and Social Media, there should be some level of integration between the two channels.  Social Media, to some extent, depends upon a strong email marketing program.

Tablets:  We generate $0.65 per customer on Tablet devices.

• Catalogs cannibalize $0.06 of demand from Tablet devices.
• Email cannibalizes $0.04 of demand from Tablet devices.
• In other words, when Catalogs/Emails are sent, customers shift their focus away from Tablet devices, and instead spend money in other channels.  When Catalog/Email activities are not happening, customers shift their attention back to Tablets.  This strongly suggests that, in the future, when customers spend more and more time with Tablet devices, this business will be able to scale back a bit on Catalogs and Email marketing campaigns, because the demand will be recaptured by Tablet devices.

Mobile:  We generate $0.15 per customer on Mobile devices.

• Catalogs cannibalize $0.03 of demand from Mobile devices.
• Email has no impact on Mobile devices.
• In other words, when Catalogs are sent, customers shift their focus away from Mobile devices, and instead spend money in channels congruent with Catalogs.  At this time, Mobile is a very tiny portion of the total story.  In the future, we need to watch this relationship, to see if it continues.  If the relationship continues, the data strongly suggest that we can cut back a bit on Catalogs, because demand will reallocate to Mobile devices.

At minimum, the attribution process should include three things.

1. Mail/Holdout Testing, where applicable.  For catalog marketing and email marketing, this couldn't be easier.  For Display Ads, this couldn't be easier.  For Paid Search, you can vary your budget and compare the impact.
2. Modification of Ad Cost by Channel.  In our example, paid search costs should be moved from the paid search budget to catalog marketing, and to email marketing.  Too often, we don't focus on this aspect of attribution.  Mail/Holdout tests illustrate why we need to do this.
3. Business Intelligence.  The goal of any attribution project should not be to just allocate demand/expense, but should be to teach every employee how channels fit together.  We just don't do enough of this, do we?  How often do we step back, and try to teach employees how our business works?

Contact me (click here) for additional assistance with attribution work.

Attribution Week: Email Productivity

If you ask 100 email marketers the following question, how many do you think will answer "yes".

• Question for Email Marketers:  Do you measure the success of email marketing by executing holdout groups, not mailing customers email campaigns for up to three months at a time?

Ask 100 email marketers this question, and fewer than five will answer "yes".

What a shame.

You see, all of the ways that email marketing interacts with the rest of your business are illustrated by email holdout tests.

Take a look at our example:

Here's the deal.  Email marketers typically measure performance via opens/clicks/conversions, adding average order size to the strategy to yield demand per email delivered.  In our case, over the course of a three month period of time. we get $4.94 demand per customer ... across 26 email campaigns (2x per week by 13 weeks) ... each campaign is generating $0.19 for a total of $4.94.

We run a profit and loss statement.

Demand	$4.94
Net Sales	$4.20
Gross Margin	$2.31
Less Email Cost	$0.08
Less Pick/Pack/Ship	$0.42
Variable Profit	$1.81

That's some sweet action!

Except, of course, that this analysis is providing us with the wrong answer.

When we do not send email campaigns to customers, as measured via mail/holdout tests, we generate just $3.33 demand per customer, not the $4.94 as measured via opens/clicks/conversions.  We know this, because in our example, one set of customers did not receive a single email campaign for three months, and continued to generate incremental demand.

Here's what the profit and loss statement looks like, with our new and more accurate style of measurement.

Demand	$3.33
Net Sales	$2.83
Gross Margin	$1.56
Less Email Cost	$0.08
Less Pick/Pack/Ship	$0.28
Variable Profit	$1.20

Now, there's nothing wrong with $1.20 profit per customer ... but it is a lot less than $1.81 per customer, right?

Let's take a look at what happens to other channels, when email marketing is discontinued for ninety days.

Paid and Natural Search results decrease, by 11% and 16% respectively.  In other words, email marketing causes searches to happen.  In theory, you have to take 11% of your paid search marketing cost, and allocate it back to your email marketing budget.  Yes, you have to do this, this is what attribution is all about.

Demand	$3.33
Net Sales	$2.83
Gross Margin	$1.56
Less Search+Email	$0.15
Less Pick/Pack/Ship	$0.28
Variable Profit	$1.13

Now, that doesn't make a difference in this case ... but when your email marketing program lacks a lot of productivity, well, it is enough to push the whole program under water.

Look at all other online marketing demand.  When you don't mail email campaigns to a customer, that customer changes behavior.  The customer re-directs demand away from email marketing, back to the website ... spending $6.57 per customer online instead of $4.35 per customer online.  In other words, customers are using email as a navigational tool to get to the website.

Look at affiliate marketing demand.  When emails are stopped, customers spend $0.24 each at affiliates ... but spend $0.34 each at affiliates when emails are delivered.  Now, the affiliate is probably getting a cut of each order, right?  Well, you have to attribute that cut on the $0.10 incremental difference, attributing that to email marketing, not to affiliates, because email marketing caused the affiliate order to happen.

Demand	$3.33
Net Sales	$2.83
Gross Margin	$1.56
Less Marketing	$0.16
Less Pick/Pack/Ship	$0.28
Variable Profit	$1.12

Display / Retargeting are not significantly impacted, are they?

Look at social media ... 28% of social media demand is caused by email marketing.  In other words, if you take email marketing away, your brand advocates don't have as much to yap about, and consequently, they spend a little bit less.  In this example, email marketing causes social media demand to happen, so you want to know that, don't you?

Email has minimal impact, when it comes to tablets/mobile.

We learned that email marketing isn't as effective as we believe it to be, when measured via traditional channels.  That being said, it's still highly profitable.

Tomorrow, we look at each channel, decomposing the role of each channel with/without advertising.

Attribution Week: Catalog Profitability

One secret to marketing success is knowing how customers behave if you stop marketing to them.


When you stop marketing to customers, you are left with brand recognition and merchandising love.


By the way, both of those things are really, really important.


So we execute a four-panel mail/holdout test.  One set of customers is treated "as usual" ... that's the catalogs+email column.  One set of customers is not mailed email campaigns ... for three months.


Yes, three months.


One set of customers is not mailed any catalogs ... for three months.

Yes, three months.


One set of customers is not mailed anything ... no catalogs, no email campaigns ... for three months.


Yes, three months.


You learn an awful lot by doing this.


We know that catalog marketing, when measured via the traditional mail/telephone channels, doesn't look very profitable.

Demand	$4.00
Net Sales	$3.40
Gross Margin	$1.87
Less Book Cost	$2.25
Less Pick/Pack/Ship	$0.34
Variable Profit	($0.72)

This is the outcome, over the course of a quarter ... it looks like we are losing $0.72 per customer ... and if you have 200,000 customers, well, that's more than $140,000 per quarter, more than $560,000 per year.


Well, that's doesn't work.  So we created the "matchback".  Any online order that happened within "x" days of sending a catalog is credited back to the catalog.  This "changes everything", as they like to say on Twitter!  Look at the table at the start of this post.  If we sum up all demand across all channels, we get $19.70, not a paltry $4.00.  Now look at the profit and loss statement.

Demand	$19.70
Net Sales	$16.75
Gross Margin	$9.21
Less Book Cost	$2.25
Less Pick/Pack/Ship	$1.67
Variable Profit	$5.29

Wow, catalog marketing works!  And that's where we quit.  This all happened back in 2000/2001/2002, our industry realize that we were "multi-channel" and we just stopped in time. We didn't innovate.  We just assumed that all of these online orders were caused by the catalog.


Not true.


Look at the results across catalog mail and catalog holdout groups.

Well, the test tells us something different.


Look at the incremental lift ... the difference between mailed results and holdout results.  We only generated $7.74 of incremental demand per customer.


In other words, if we don't mail any catalogs, the customer continues to generate 57.9% of the demand the customer was going to generate anyway.


I know, you don't want to believe this, you want to believe that customers crave catalogs.  Well, these days, customers are influenced by customers --- demand will still happen, regardless.


What does the profit and loss statement look like?

Demand	$7.74
Net Sales	$6.58
Gross Margin	$3.62
Less Book Cost	$2.25
Less Pick/Pack/Ship	$0.66
Variable Profit	$0.71

Ok, there's nothing wrong with this.  But the outcome is not what matchbacks suggest.  Instead, the results are 58% of what matchbacks suggest.


In this case, your job is to discount your matchback results, multiplying whatever your vendor tells you by 0.58.  Then, you run your profit and loss statement on what remains.


Later this week, we'll dig into the specific channel-based results, showing what happens to other channels when you stop mailing catalogs.