Pick Up A Broom

A little bit of a parable for you.

In my community (a retirement community though I'm not retired, which of course is a goal I will one day achieve), I am the Master of Ceremonies for a 200 person party. Catered meal, I facilitate the program. Every year there are maybe 25 people out of 200 who don't like the party. They have "ideas". 

• "Have you thought about serving pheasant, because people love easting pheasant."
• "Why don't you host the party outside? I mean, so what if it rains?"
• "The bar ran out of Pabst beer, and that's on you, that's your fault, you ruined the party."

I'm on a party planning committee. Each year maybe 3 of the 25 grumblers "wants to do something about it". They ask to join our committee, we say yes, we host a post-mortem of the party from the prior year, the three new individuals air their grievances, we listen, we tell the new people that maybe they could be the ones who fix the problem. If there isn't enough Pabst Beer, is the new person willing to work with catering to solve the problem?

What do you think happens when the person raising the issue is asked to solve the problem?

Yeah, the person quits the committee.

The post mortem on the United States will show that Fox News, CNN, and MSNBC destroyed the country from within ... creating a disincentive for those who "do" the work ... creating fabulous incentives for those who criticize those who "do" the work.

Are there any parallels in retail and/or e-commerce? Maybe you listen to a podcaster who has ideas about how to fix JCP. Why don't you (the podcaster) go fix JCP? Put your reputation on the line and see if you are an Actual Leader or just a Thought Leader.

My wife worked with an HR leader who used to say "Pick Up A Broom" when somebody would criticize. In other words, when somebody accidentally empties a box of Special K cereal on the floor you can point that fact out to somebody, or you can pick up a broom and clean up the mess yourself.

P.S.:  Ask me sometime what it was like to run pickleball ratings for a year at my pickleball club. There were plenty of incentives for those who criticized people who "did" the work.

Returns

As J. Peterman said on Seinfeld ... "well that looks like a lot of words"

The omnichannel thesis loves free returns. 

Have you ever actually analyzed how customers who return merchandise behave?

First, assume you have three customers.

• Customer #1 = Spends $100, Keeps $100.
• Customer #2 = Spends $150, Keeps $100.
• Customer #3 = Spends $150, Keeps $150.

Which customer is most valuable?

This one is a fun exercise. It generally requires a bit of regression nuance to parse out the value of the $50 that is returned. In most projects, Customer #2 is more likely to repurchase in the future than Customer #1 but less likely to repurchase in the future than Customer #3. In terms of value, Customer #2 is less valuable than Customer #1 because of the costs associated with the return.

Second, there is a limit where returns behavior becomes punitive. A whopping thirty-two years ago at Lands' End we used the Hyperbolic Tangent Function to model returns behavior (we used this function because we had customers who returned more merchandise than they purchased ... ponder that one). We learned something interesting (I shared this with you previously).

• Any customer purchasing at least three times and returning at least 70% of what they purchased was a customer we didn't want to market to anymore, because that customer would return 60% of future merchandise ... the relationship would not be profitable.

In 1993 that was a big deal ... it meant that the 13-50 catalogs this customer would normally receive were reduced to maybe four. And wowzer, did the customers who went from 13-50 catalogs per year to four HOWL. I caused our call center employees a bunch of grief.

In 2025, why does the high returns customer deserve ten (10) email campaigns per week? Why not cut that number down from ten to two? You are not stopping the customer from buying from you ... you are simply cutting back on marketing to the customer who wants to return stuff.

Perform the analysis, and do something!

Parsing Words

This one is filled with goodies.

Among the words/phrases used to smear actual meaning:

• Leverage
• Unifying Campaign
• Global
• Cultural Activations
• Emotional Core
• Shared Human Experience
• Powerful Anthem
• Movement
• Resonating
• Story-Rich
• Resonance

There is no mention that sales actually increased. There is a reference to an "18% increase in purchase consideration", whatever that means.

Granted, the purpose of many marketing campaigns is to create awareness - I myself preach it.

But at some point somebody must buy something.

I recall working at Nordstrom ... one year we spent something like $15,000,000 on Fashion Week in NYC. We asked our customers to download a program (pre-app days) to keep up with all the action that week (our marketing agency came up with this idea). I don't remember the exact numbers, but a few thousand customers downloaded the program ... not a failure but certainly an expensive way to burn money.

I recall performing the analysis of our Fashion Week campaign, thousands in sales against fifteen million in cost. I recall my boss (the Chief Marketing Officer) saying "your analysis will never see the light of day". Instead we used phrases designed to smear the real meaning ... stuff like "resonating" and "powerful" and "community".

A trade journalist once told me that "everybody wants to read a case study but nobody wants to share a case study." There is a good reason for that. If something truly worked, why would you give a blue print to the competition?

Of course, this means you instead get to read all about stuff that resonates with the "emotional core" of the customer.

Won / Lost Record

I've told you this story several times, but it really speaks to the importance of "how" you communicate.

More than a decade ago, I had a client that performed terribly. Sales were in decline, profit was tepid. And yet, you'd visit their campus and "all was good". Nobody, and I mean "nobody" cared.

So, I converted their high-level financial data into an NFL Won/Lost record. I created an equation based on financials for publicly traded retailers - then plugged their data into the equation. It looked something like this.

In this case, the business contracted by 6.7%, and posted a 4% pre-tax profit rate. This translates to an NFL team going 6-11.

Well, I compared their brand to the home town NFL team, a team that had a comparable record the year prior, firing the coach in the process.

This DID ... NOT ... GO ... OVER ... WELL!

"We aren't that bad!"

"Are you suggesting we should be fired?! Really? You think WE should be fired? We're paying you for your advice, and you do this?"

Within a few days, the brand converted a conference room into a "War Room". Within a few weeks, the CEO fired the Chief Marketing Officer. Problem solved! (Narrator:  The problem was not solved).

One of the areas I catch grief for these days is email marketing. I'll state that 20% of e-commerce sales should come from email marketing. Clients HOWL at that suggestion.

• "You obviously don't know about attribution ... email is a bottom-of-the-funnel channel, you're wrong."
• "You don't understand, our ESP said if we mail too often we lose customer trust and we'll be penalized and be placed in junk mail folders."

And yet ... regardless of attribution solution or ESP "guidance" ... clients with a high fraction of sales from email marketing are generally highly profitable.

So instead of arguing ... maybe look at your metrics and then look in the mirror, ok?

Omnichannel!!!

Your industry is run by corrupt imbeciles (click here).

I recall being at NEMOA years ago - the speaker (a catalog co-op executive, one of the people many of you view as a 'trusted partner') walked off the stage, stood next to me, and said "these people are so dumb, they'll do anything I tell them to do". He was talking about marketing attribution, of course. And he was talking about you.

It's like the paper rep who once called me when he learned I was cutting the catalog circulation budget by 20% (I worked at Nordstrom and we were losing $30,000,000 profit on sales of > $300,000,000) and said "I'm sorry, I can't let you do that." For a moment, I became Dr. Reston on Seinfeld ("you can and you will").

Too often, your 'trusted partners' are people looking to scam you and your company so they can have the good things in life. Their happiness and satisfaction come at your expense.

Email Subscribers: Clicks per Year

How many times per year should an email subscriber click through a campaign and interact with your merchandise?

Ideally, it's a big number. Intuitively, it's a big number.

In reality ... it's two (2). Two times per year.

That's in-part a function of how tepid and feckless most email marketing programs are. I don't need to know for the 20th time this month that you are at 50% off plus free shipping, with no product ever featured above-the-fold. You are not even trying. No wonder nobody clicks through your campaigns.

If you do an average job ... just an average job ... some of your email list will click-through at least two campaigns per year.

Go run the query right now ... find out what percentage of twelve-month buyers with a valid email address clicked through at least two times in the past year? Get ready to be disappointed with what you learn.

Are Search Customers New Customers?

Ha!

There's folks out there who tell you that, when evaluating channels like Search or Social, you are evaluating revenue from new customers divided by total ad cost in those channels ... you're assuming that those channels deliver new customers.

And yes, they deliver new customers.

But your existing customers also respond to those channels.

Here's what the data looked like for one brand - percentage of search orders by customer life stage.

• 20% = First Order.
• 10% = Second Order.
• 20% = 3rd - 5th Order.
• 18% = 6th - 10th Order.
• 32% = 11th+ Order.

Yeah, that's actual data.

Oh oh.

This, of course, renders many of the canned aMER calculations useless.

More important, what does it mean if 32% of search orders come from customers ordering for at least the eleventh time with your brand? I mean, they're loyal to your brand or they wouldn't have purchased ten or more times ... and still, they're out there searching, looking for a different dating partner.

Your answer to that question tells you a lot about the value proposition of your brand.

P.S.:  The higher your annual rebuy rate, the higher the fraction of search customers who are well-along the customer life cycle. The lower your annual rebuy rate, the more the skew of search customers to new customers.

Feedback

Another birthday ... the years are adding up, folks. I won't be doing this work forever!

Sometimes I look ahead ten or fifteen years. A world where I'm playing pickleball even more often. Would anything that has been shared over the past two decades still be used? Remembered? Or will it just be you and your AI companion employing best practices?

Do me a favor. Send me an email (kevinh@minethatdata.com) and share some feedback.

• Your favorite post(s) from the past two decades.
• How you are using this content in your job - have you employed any tactics that benefitted your brand?
• What would you like to see me write more about in the future?
• Who do you hire that competes with me, and why do you make that choice?

No replies are bad replies ... let me know your thoughts.

Scrubbing The Humanity Out Of Things

This came across Bluesky on Monday. This was a gif ... the photos represent the start of it and the end of it.

Now I'm not a fan of driving any brand off of any website, that's insane.

But the AI-based sentiment is reasonable. I mean, look at how sterile the second image is.

I get a lot of emails ... some of them are those ridiculous "pitches" from vendors ... they're using AI to "speak" to me. They are also scrubbing the humanity out of the processes they're recommending AI for.

There is a future date where AI data centers consume all of the electricity and water that we deserve. Between now and then, be sure to balance your ability to do more things without scrubbing all of the humanity out of what your brand does.

An Exception to Sloppiness

I had a client with an approximate $60 AOV. That client couldn't get away with anything. Absolutely anything. Any level of sloppiness was met with p&l challenges. When you have a low AOV, you have to have a high attention to detail to make the business work.

In B2B, you might have a high AOV ... it's common to see $2,000 orders or $4,000 orders. Do you know what happens when you have a $3,000 AOV? Sloppiness. You can get away with any level of marketing expense mismanagement, because you are generating $1,800 of gross margin per order. Make a ton of mistakes? $1,800 of gross margin per order might go down to $1,750.

Meanwhile, the same $50 level of sloppiness puts the $60 AOV brand out of business.

If you are a marketer, you might be amazed at the level of discipline (or lack of it) that the B2B marketer with a $3,000 AOV possesses when you switch jobs from a low AOV brand to a high AOV brand.

If you work for an agency or are a consultant, you intuitively know this fact. Your job is much harder convincing the $3,000 AOV B2B brand to do anything that improves the p&l. The agency pro needs to know the audience the B2B brand speaks to before determining whether a B2B brand is smart or not.

It Costs 5x More Blah Blah Blah

Over on LinkedIn, the thought leaders were arguing about the importance of keeping a customer.

Pure, unadulterated thought leadership.

One of the gurus had to go there ... couldn't stop herself ... 

• "It costs five times as much to win a new customer as it costs to keep a customer."

It's easy to measure if somebody truly understands marketing ... if they tell you it's cheaper to keep a customer than to "win" a new customer they don't understand marketing or business.

Let's walk through an example, because this is how this stuff works in the real world.

Say you have a company with the following dynamics.

• 10,000 customers and a 25% annual rebuy rate.
• 7,500 new/reactivated customers per year.
• Your digital marketing budget and your loyalty marketing budget are identical.

Your marketing manager reads drivel on LinkedIn and decides to be "strategic". He cuts the digital budget in half, he takes the money from the digital budget (mostly new customers) and spends it on additional loyalty efforts.

The money spent on additional loyalty efforts "works" ... rebuy rates increase from 25% to 33%. SEE - IT WORKS! Of course, new/reactivated buyers tumble by 25%, but that was expensive stuff and it was, as LinkedIn says, "costly".

Base Case:  10,000 buyers * 0.25 rebuy + 7,500 new/reactivated = 10,000 buyers next year.

New Idea:  10,000 buyers * 0.33 rebuy + 5,625 new/reactivated = 8,925 buyers next year.

Somewhere there should be alarm bells going off, but LinkedIn is happy and that's all that matters.

Year 2:

Base Case:  10,000 buyers * 0.25 rebuy + 7,500 new/reactivated = 10,000 buyers next year.

New Idea:  8,925 buyers * 0.33 rebuy + 5,625 new/reactivated = 8,570 buyers next year.

LinkedIn once again celebrates. The CEO is asking questions, questions like "why are we spending the same number of marketing dollars and yet the business is contracting?"

Now it is Year 3:

Base Case:  10,000 buyers * 0.25 rebuy + 7,500 new/reactivated = 10,000 buyers next year.

New Idea:  8,570 buyers * 0.33 rebuy + 5,625 new/reactivated = 8,453 buyers next year.

By this time either the CEO or Chief Merchant is about to be fired, and neither one wants that so they decide to fire the glib marketing manager who hops on to LinkedIn looking for a job as a "strategic marketer" who knows how to "optimize budgets" while embracing "loyalty marketing". AI filters allow this resume through the door because it clicks all the right boxes.

And yet? The marker is a Lemonhead. The marketer killed a business and didn't even understand what happened.

Don't be a Lemonhead.

Undercounting Email Marketing Performance

When I'm asked to analyze email marketing performance, it's common for the email marketing professional to share opens / clicks / conversions. Good stuff, no doubt! You can tell how much companies care about email marketing based on how good of a job the email analyst does explaining what is happening. Most of the email analysts I've met are darn good at their job!

In fact, most of you are darn good at doing your job. You are most certainly not Lemonheads!

Exceptional email marketers do three things that set them apart from everybody else.

1. They are brilliant communicators / evangelists.
2. They frequently execute holdout tests and consequently they know more about the value of their channel than anybody else knows.
3. They measure "unconverted visits" and know the value of a visit.

(3) above is a classic example of "not" undercounting email marketing performance. Nearly everybody else undercounts email marketing performance.

What do I mean by "undercounting email marketing performance"?

• When a customer visits your website and does not purchase something, the very act that the customer visited your website has "value", and that value is undercounted.
• In other words, when a customer visits and does not buy something, the customer now has a "visit recency" of zero months, meaning the customer is significantly more likely to purchase in the next thirty days than is a comparable customer who did not visit the website.

All of you understand the importance of old-school "RFM" segmentation ... customers who bought from you 0-3 months ago are more valuable in the future than are customers who bought from you 22-24 months ago.

The same concept holds for "visit recency". Visits in the past thirty days have significant value to your business. You want people visiting your website ... often!

I have many clients who do a fabulous job of keeping customers coming back to the website to shop ... some are really good at this by using email marketing to advertise "product scarcity" ... if you don't buy "product x" it will be sold out soon is an example of leveraging product scarcity to get the customer to your website.

Anyway, as you go from being very good at measuring email marketing campaigns to being brilliant at measuring them, you'll stop undercounting email marketing performance by measuring future sales caused by each incremental website visit caused by email marketing.

Price Bands

One of the things I'll be looking at in this run of the MineThatData Elite Program is price bands.

Many businesses have items priced, say, in the $10.00 - $19.99 price band. This ends up being a high-volume price band that is responsible for a ton of customers.

Well, you toss tariffs into the mix and now that $19 item might cost $22. It's in a higher price band.

It's common for sales to increase and customer counts to decrease in times of inflation and/or increased cost of goods. This "can" cause long-term changes to the business. Vacating a price band can cause an audience to vacate as well. And yes, the opposite can happen ... customers move up in a price bracket and do not change behavior.

It's important to understand how your customers adapt and adjust, right?

P.S.: You measure the average price per item purchased (after discounts/promos) for customers purchasing from email campaigns, and you compare the metric to other channels, right? Hint - you need to do this. Your email customers are "different", and oftentimes it's your fault they are different.

Where Did The Money Go?

Something is going on ... (click here).

Speaking of private equity, I worked a lot of projects from 2012 - 2018 with private equity folks. They wanted to understand how much business would still happen if catalogs were scaled back or didn't exist. It was always interesting that they wanted the answer but actual catalogers didn't want the answer.

Post-COVID, those projects ended. Once you know the answer, no need to pay to get similar intelligence.

Worse (for me, for monetization purposes), you quickly learn a secret, a hack, one that allows outside investors to avoid me altogether.

• If a business is still generating 15% or more of sales via a call center (i.e. customers phoning in an order, talking to a live voice), the business will have a challenge escaping catalog marketing.
• If a business generates 20% or more of sales via email marketing, the business can escape catalog marketing.

Once you see that pattern repeat, no need to keep paying for knowledge.

Also interesting - you have to be all over the finance folks. In my client work, it's common for Finance folks to be great business partners. Very common. But every once in awhile you run across a situation where things just don't seem right. Those would be cases where, if private equity is involved, private equity needs to provide adequate oversight, so nobody ever has to ask "where did the money go?".

"I Don't Like This Business Model"

Catalogers telling Amazon they are "doing catalog wrong" has me thinking.

Let's go back to the end of the catalog era at Nordstrom (which, as it turned out, was the beginning of the end of catalogs, period). My team tested the living daylights out of catalog mailings, and at "best", the entire $160,000,000 endeavor was a break-even proposition. Why would you generate $160,000,000 of sales that generates $0 of profit?

Yes, I get it, the market share gurus will retort. Market share folks don't always have to worry about the uncomfortable constraints of profitability.

A decision was made to create a new "catalog", one driven by our retail marketing team. There would be a monthly catalog, and for $27,000ish a vendor could purchase a spread and advertise their products. Circulation = 2,000,000 (by the way, the holdout test sample was 200,000 customers ... yeah, 200,000 ... and we sure did learn stuff by having a proper holdout test in each mailing ... we could slice-and-dice as we wished).

People with a catalog heritage thought this strategy was an abomination.

"They're doing it wrong."

"That crap will never sell."

"You can't have a hodge-podge of creative shot by individual brands and then cobble it together recklessly, the presentation won't be cohesive."

"You're letting the foxes run the hen house."

"This is damaging to the brand."

Catalog staffers suffered through the first in-home date, mocking the abomination as it reached mailboxes all over America.

Sales results tricked in ... and the word "trickled" was appropriate. The new catalog generated 1/5th the sales that the discontinued catalog generated.

"I told you these fools have no idea what they're doing."

One problem.

It was my job to run the p&l for the "abomination", and compare it to a comparable group of customers receiving the old-school catalog that the catalog professionals loved. These are actual-ish results, right here, at a comparable customer segment level.

The old-school catalog would generate $3.00 for every catalog mailed (on average). We'd run the p&l and show that we were getting $0.09 profit per catalog mailed (this was for a comparable segment of customers that also received the new "co-op" funded catalog ... not the break-even proposition of the catalog in total).

Please read down the "Co-Op Catalog" results column.

Which catalog was more profitable?

The "abomination" was more profitable!

No matter how I looked at it, no matter how many abominations we mailed, the no ad-cost co-op funded version was more profitable.

Every time.

In fact, at $0.08 per book across twelve mailings across 2,000,000 in circulation, the abomination was $1,900,000 more profitable on an annual basis. The tactic drove less top-line sales and more bottom-line profit.

And with that, the catalog professionals jumped ship.

One by one, they were gone. They took their "parting shots" on the way out the door, telling people how "stupid" the decision was (to eliminate the old-school catalog and replace it with an abomination that was more profitable). 

One of the final phrases my Circulation Director issued before jumping over to the e-commerce division was this sentence.

• "I don't like this business model".

Finally somebody was honest! Thank God!

It's ok to not like a business model. Leave. Go work for a business model you appreciate.

It's also ok for a company (like Amazon) to execute things in a way that you might find sloppy or inappropriate. People generally don't want outside experts to fix things. Do you think the professionals at a catalog agency want outsiders coming in, telling them that "you need to pivot to AI?" Cause that's where we are in 2025. An outsider would look at a catalog agency and say "you're doing marketing wrong". The outsider would be right. And the outsider would have zero chance of being hired by the catalog agency.

Apparently Amazon is "Doing It Wrong"

Over on LinkedIn, catalog agency folks and paper gurus are frustrated with Amazon ... they are criticizing Amazon for failing to "target" the recipients of the catalog properly ... openly soliciting Amazon for business in the process. Nobody ever executes catalogs "the right way" for these people.

I asked CoPilot to create another cartoon. Yes, AI has a spelling error ... extra credit for those who catch it ... but I want you to read what CoPilot added for text after drawing the image.

Email Test

Let's give you the same offer I made for catalog mail/holdout tests.

• For $900 per test, I'll analyze your email mail/holdout tests, showing you how customers interact with other channels when email doesn't exist.

Let's prove that your email marketing program has value (hint - it does) ... heck, you might learn that the majority of your e-commerce division's profitability is entirely due to email marketing (that's what many of my clients learn).

Contact me now (kevinh@minethatdata.com).

What Does An Email Mail/Holdout Test Look Like?

More than twenty years ago at Nordstrom we personalized every email we sent, and we executed mail/holdout groups. We knew that email was more productive than what our open/click/conversion metrics showed - we knew it was worth the marketing investment to personalize all of it and then generate another 20% on top of something that was already wildly profitable and not getting full credit.

We did that back in 2002. That's twenty-three years ago. Think about that for a moment.

You're emailing customers 10x a week, so you could hold out a group of customers for three to fourteen days, learning how customers interact with all of your channels.

After a few weeks, you average demand/sales by attributed marketing channel.

This is a very common outcome in an weekly email mail/holdout test. The two numbers to focus on are bolded in blue above.

• The group that received emails all week spent $0.50 per customer. This is what you'd normally report across maybe 5-10 campaigns, in total.
• The group that did not receive emails all week spent $2.00 - $1.54 = $0.46 per customer of incremental volume.

In other words, 92% of what this company commonly reports as email marketing revenue is actually email marketing revenue ... the other 8% would happen anyway if email marketing campaigns were stopped.

It's common to work with clients where email marketing might account for 15% to 20% of annual volume. That's a lot of volume! In this case, we'd multiply 15% to 20% of annual volume by 92% based on the test result, which means that email marketing is truly responsible generating A FORTUNE!

I've been in meetings where we are able to show that a company with seven percent pre-tax profit would be unprofitable without email marketing. You should see the eyes on the Finance folks when that fact is shared. And yes, somebody then says they can't believe the science behind a mail/holdout tests though they sure seem to appreciate the math behind clinical trials.

Email marketers almost never get the credit they deserve. A small number of under-resourced individuals do all this work and then present the findings via opens / clicks / conversions, which are metrics that your average employee struggles to be passionate about. Profit? At worst case, your Finance team becomes interested.

I've been in the meetings where the email service provider tells my client not to execute tests ... "oh you don't want to do that!". Why not? Seriously. Why does a vendor get to tell you what you should and should not learn (I know, you're going to say why does a rogue consultant get to tell you what you should and should not learn).

Did you know that when I was VP of Database Marketing at Nordstrom, back in 2003, my testing budget for marketing channels was 3% of annual e-commerce net sales? In other words, if we were a $400,000,000 e-commerce channel, we were willing to forego $12,000,000 in annual sales to learn how our customers interacted with retail stores, our website, and our catalog. Hint - that's how we learned we didn't need to mail catalogs anymore.

You want to prove your value to your company and then secure resources to do the stuff you've always wanted to do that nobody will give you money to execute because the people who have the money don't align with opens / clicks / conversions.

I worked with a client a decade ago who had an email professional who personalized all of her campaigns and generated what appeared (via tests) to be a 50% increase in the productivity of her campaigns. She didn't take the logical step to convert her numbers to profit ... so when she left the room one of the Executives looked at the Executive next to me and said "WHAT A NERD".

That's what you are up against. That Executive likely earned $250,000 a year plus an 80% bonus that allowed him to purchase a new BMW every year if we wanted to do so ... all because this young woman was generating most of the profit at her company. And he had to belittle her ... she's the problem.

No. She's the reason the brand was profitable.

Execute an email marketing holdout test.

Let's Try Something!

For the low cost of just $900, I will analyze your email mail/holdout test or your catalog mail/holdout test. I will show you how your catalog housefile matchback rules should change. I will tell you what your organic percentage is. I will tell you how profitable your catalog effort was. If it is an email mail/holdout test, I'll tell you if your email marketing program is being over-valued or under-valued.

For $900. Per individual mail/holdout test.

You have one (1) day to take me up on this offer. kevinh@minethatdata.com.

Go! Learn how your customers interact with your business. This stuff is fun!

Reader Question

This one came up numerous times last week.

Question:  "Why in the name of Don Libey are you talking about catalog mail/holdout tests? Seriously! It's 2025. I executed mail/holdout tests at Fingerhut in 1988. What is wrong with you?"

Allow me to tell you a story. I spoke at a conference a few years ago. I was asked to share what I've learned from executing and analyzing a thousand or more mail/holdout tests over a thirty-five year career (and not just from catalogs - email mail/holdout tests - search geography-based on/off tests - display on/off tests, you get the picture).

I mean, it might be thousands of tests I've been associated with, each including thousands or tens of thousands of customers per test ... tens of millions of data points. I doubt there is any person on planet Earth who executed and analyzed more tests applied to a print-centric business than I have. 

So I shared what I learned. I could tell that the audience didn't like what I revealed. Long faces. Heads shaking "no" left-to-right and right-to-left. People not taking notes.

What I was sharing, of course, was the fact that the attendees were on the precipice of "re-inventing" their businesses. The audience didn't want to learn that. The audience wanted to learn that they could keep doing what they loved doing.

I finished my presentation ... tepid applause. A paper professional immediately pulled me aside and told me he couldn't let my message ever get to his clients because my message would, and I quote, "take food off of his dinner table".

That's when the next speaker took the stage ... a person representing survey results from a print-centric agency. The speaker immediately attacked my presentation. He pointed to me in the distance, from the stage ... "He's quoting tests. Tests have all sorts of error and variability associated with them. I'm about to share facts with you. Yes, facts! We surveyed actual customers, and we know the truth." The audience bubbled with enthusiasm, as the speaker shared the results of a survey with a few hundred respondents ... a survey that was clearly designed to communicate the intangible value of print.

At the end of the presentation, the audience vigorously applauded. They were happy!

I realized I had been "set up". My presentation was there to communicate a point of view (the truth), but was then followed by an hour-long message that the conference organizer wanted the audience to hear. Within an hour, the audience perspective was reinforced. #printisback. All based on a survey of three hundred individuals. Positive vibes. Three hundred individuals were valued more than the millions of customers I'd analyzed via mail/holdout tests.

I analyzed a mail/holdout test for a client, a test showing that the catalog was wildly unprofitable. Their agency pushed back, suggesting that profitability wasn't the right metric, that I should be looking at the fact that engagement was significantly boosted (it was - website traffic increased nicely).

It's October 2025. Business owners and Executives cannot run businesses off of vibes and print engagement. Social engagement? Yeah, have at it, costs are minimal. Print is held to a different standard due to the prohibitive costs associated with print.

Execute mail/holdout tests. Learn. Adapt.

How Do Digital Channels Interact With Catalog Marketing Efforts?

Here's the test result we've discussed this week.

There is so much to digest within the table. The impact of catalog marketing on digital channels can easily be observed in the table. Your attribution rules clearly come into focus here.

Email Marketing:

• The catalog essentially has no impact on email marketing. About 4% of email demand is "caused" by the catalog.

Search Marketing

• A different situation. Half of Search Marketing demand is caused by the catalog ... the other half of Search Marketing demand happens independent of the catalog. Since the catalog causes half of Search demand, the catalog should get credit for the demand it pushed to Search ... and ... and, the catalog should be penalized for the ad cost in Search that wouldn't have to be spent had the catalog not been mailed.

Paid Social

• A similar story here. 60% of Paid Social demand still happens if catalogs are discontinued. The catalog should get credit for 40% of Paid Social demand and should have to absorb 40% of Paid Social expense.

The results of the test inform your attribution rules. Not just inform them, they determine your attribution rules!

So, that concludes our study of this one mail/holdout test. If you believe in doing the right thing for your company, you'll keep pushing this rock uphill. It will be an exhausting experience. It will be a worthwhile experience - protecting your company well into the future.

Question? (kevinh@minethatdata.com)

What Does It Mean?

Here's our test results.

We know that the catalog isn't generating profit as a whole. Now, if you run a dozen tests like this and get comparable results, you have a series of decisions to make, don't you?

At a segment level, your matchback results consistently lie to you.

At a segment level, after taking credit for just 30% of what the catalog appears to generate, you can't believe your own eyes, can you?

Each row represents a customer segment - the segment via matchback reporting (the Matchback column) converts demand to profit (the MB or Matchback Profit column). Only the segment generating $1.00 per book is unprofitable. That's the world many readers live in today.

However ... look at the "Test True Gain" column. Here we only take credit for the 30% of demand that the test tells us we can truly take credit for. Fewer than half of the segments are profitable.

Instead of being able to mail nine out of ten segments ... we can only mail four out of ten segments. Circulation would have to be cut by 1 - 4/9 = 55%.

This is the point in the discussion where catalog professionals become angry with me.

• "We can't cut circulation by 55%, why are we even mailing catalogs at that point?" Good question!
• "If we cut circulation by 55%, we'll lose sales, and we can't contract the top-line. No smart company achieves success by getting smaller".
• "What if you are wrong, Kevin?" Nobody ever wants to answer the question "What if the catalog professional is wrong?"

From here, there's all sorts of obscurification. 

• "We have to retest". 
• "I just don't believe you".
• "I'm not laying off half of my call center just to make you happy."
• "My catalog marketing agency told me to not listen to you, that test results are not a best practice given that matchback analytics have been around for twenty-five years."
• "My paper rep said he'd give me a discount to entice me to keep mailing catalogs. And the USPS is running a 10% off promotion. I'd rather band together with other catalogers and do what is right."

All of that, of course, ignores the reality of the situation.

• The catalog is not generating profit.
• The catalog professional could make decisions to make his/her company a fortune.
• What would the CFO think if she knew that the catalog professional was purposely not maximizing profitability?

Tomorrow, we'll wrap up this mini-series with a discussion of digital marketing channels.