Really, Really Bad Models

Two things about models, for those of you considering a QuickScores project.

First, industry vendors attack me (I know this because you tell me they do this when you meet with them) by saying that my models are too simple ... there are only a handful of variables and the techniques (logistic regression, ordinary least squares regression) are "old school". They attack, of course, because they're trying to sell something complicated. When they attack me, ask the attacker what s/he thinks about the term "parsimony", because if the person is a credible stat expert they know about the importance of building a model with as few variables as possible. If they understand the term and the meaning of parsimony, ask them why they are trying to sell you something that is more complex than necessary?

Second, just because a person is building you a model doesn't mean that the person has any clue whatsoever what they are doing. I don't care that they've been employed by "Vendor X" for the past four years and have worked with all of the "Leading Brands". Why suggest this? I've told you the story ... sitting in the Executive Conference Room at a "major brand". On one side of the table was the vendor, saying that their model and 1,033 variables (it was more than a thousand, yes ... more than a thousand) was the best option for the brand. On the other side of the table was a PhD researcher hired by the "brand" to "in-house" math-related stuff. His model was reasonable ... maybe 10-15 independent variables ... but his dependent variable was complete nonsense. He was predicting who was going to buy from the brand, not who was going to buy from the catalog. I asked the researcher why he didn't calibrate the model toward A/B style mail/holdout tests, tests that clearly showed that retail buyers had NO INTEREST in catalogs whatsoever and therefore shouldn't be included in any circulation plan? The conversation went something like this:

• Researcher:  Are you actually questioning me?
• Kevin: What?
• Researcher:  What gives you the right to even question me or my credentials?
• Kevin:  Because you don't know what you are doing. You have mail/holdout tests that clearly tell you that 80% of your customer base could care less about catalogs and shouldn't be mailed. Why are you building models that will prioritize those customers?
• Researcher:  You clearly know nothing about building a brand.
• Kevin: What you are doing will cause you to generate less profit, thereby harming your brand.
• Researcher:  I'm mailing who I want to mail, and those will be customers who are loyal to the brand.
• Kevin:  Do you agree that if a customer won't spend incremental dollars because of catalogs that the customer shouldn't receive a catalog?
• Researcher:  No.
• Kevin:  Why not?
• Researcher:  Just because I don't. This conversation is over. I swear, you don't know anything about math, and "Vendor X" really doesn't know anything about math.

When the CFO asked me who to believe, I told the CFO to believe me.

The CFO again asked me which party (vendor or in-house employee) to believe? I said "neither".

I was not asked back for a few years.

Here's the problem ... if you aren't trained in statistics ... and you don't need a ton of training ... you don't know ... you COULDN'T know ... that you are being bamboozled by an "expert". There are times the expert doesn't know that the expert is clueless.

Our industry uses a lot of really, really bad models. The bad models cost us sales, and cost us profit.

Give a QuickScore a try. And if that's not the direction you want to go in, no worries. But then please figure out how the heck you are going to vet the experts when you don't have the skills to vet the experts.

We've Used The Same Model For 12 Years. It Still Works!!

This topic comes up in many "QuickScore" project proposals.

You hear the comments ... "We built our matchback algorithm in 2007 and we're happy with it and we aren't going to do any print mail/holdout testing because if we did that we'd lose sales and who wants sales to decrease? We are using an RFM+C segmentation strategy for print, and we established that back in 2008. It still works!! Everything is great. So we don't think that spending $8,000 on QuickScores is a smart idea, but we wanted to hear about your methods just in case we wanted to change something internally, thanks."

Mind you, a company like this spends $750,000 a year with catalog co-ops and grumbles all the way to oblivion wondering why those names perform at 40% of the level they performed at 10 years ago, but whatever, $8,000 is just too much to spend to modernize.

I'm no different. My car is nearly 13 years old. It still works!!

Well, there is a big difference. I've driven a lot of rental cars, and there are very few that drive as well as the 13 year old car I own. And I've only had to spend $1,800 fixing the car in the past decade. 

Meanwhile, you with the "same model you've used for twelve years" ... have you had a chance to see what other models look like when applied to your business? How do you know that what you are doing "still works"?

How do you know that what you are doing "still works"??

Last Chance for This Run of the MineThatData Elite Program

In this run, we're going to look at the impact of COVID on your business. This high-level analysis focuses on the style of metrics I use in more-detailed analytics projects.

Cost = $1,800 for the first run, $1,000 in each subsequent run, future participation is fully voluntary.

Five years of data, one row per item purchased.

Data is due by June 15.

Payment is due by June 15.

Analysis will be completed by June 30.

Training The Customer

We use price and promotion to train the customer how to behave.

It's hard to analyze a first purchase or a second purchase and learn anything about customer pricing preferences. But by the time the customer purchases for the third time, behaviors become solidified.

For many of us, it takes 6-18 months for a customer to get to a third purchase. During that time the customer will see 300 email marketing messages. The customer will know our promotional cadence. The customer will not believe that 20% off is relevant, because the customer has 60 examples of cases where you lied about 20% off and then offered 40% off. The customer will wait until you buckle.

I've mentioned this test numerous times ... in 1998 at Eddie Bauer we promoted all customers who had not purchased in three months. They all got 20% off ... or more. Then we noticed that purchase response had unnatural "bumps" at three months. So my team executed a test. We did not promote customers for six months, period. What happened?

1. Customer response dropped dramatically for customers with recency = 1 month / 2 months / 3 months.
2. Customer response increased significantly for customers with recency > 3 months.
3. After six months, there was no difference in response or spend. There was, however, a huge increase in profit among customers who were not promoted.

We train the customer to behave the way we want the customer to behave. There are external factors (i.e. lower relative income) that we have to react to, of course, but we play a role in training the customer.

So let's train the customer appropriately, ok?

Deflationary Trends

When you read that Nordstrom generated 60% of sales in the prior fiscal year (pre-COVID) from ONLINE and RACK, you realize that something very different is happening among their customer base (click here).

Think about it this way.

• In 2000, the customer would have spent 85%ish of sales in Full-Line Stores, 10% in Off-Price Rack Stores, and 5% via Online/Catalog.
• In 2019, the customer spent 40% of sales in Full-Line Stores, maybe 35% in Off-Price Rack Stores, and maybe 25% Online.

So you've got a fundamentally different dynamic that you are managing, right? While this isn't mathematically fair to say, it's like there has been an implosion of the Full-Line Store concept resulting in a third +/- of all traffic to disappear. It's not a third, that's not how the math works, but for illustrative purposes it should cause you to think, and that's what I'm asking of you, right?

A company like Nordstrom builds two separate business units ... growing one while the other sets up for contraction.

Dollar stores are different ... they sprouted up and continue to expand as the middle class is whittled away. They built a customer base because their customers didn't have a choice ... less relative income, a lack of quick access to a Walmart. Dollar stores go up, mall-based stores die.

Now we turn to your brand.

The most common deflationary trend surrounds margin manipulation.

• 2000 = Buy an item for $19.99 ... cost of goods is $9.00, gross margin = $10.99.
• 2020 = Buy a comparable item for $24.99 ... at 30% off ... cost of goods is $6.50, gross margin = $10.99.

See what happened with margin manipulation?

• The customer paid $17.49 ... not the $19.99 the customer paid in 2000. If you adjust for inflation, well, the customer really paid $11.77 in 2000 dollars. Similarly, you made less money after adjusting for inflation ... but on the surface, you made comparable gross margin dollars.
• Your suppliers got squeezed. In return, your suppliers had to squeeze somebody, which leads to an awful lot of shoddy quality and lowly-paid employees outside of the United States.

Think about whether this dynamic happens at your brand or not.

The Deflationary Customer

During the past two months, we've talked about customers who warrant QuickScores.

• Print-Centric customers who can warrant MORE mailings ... compared to the vast majority of your customer base who don't need any print whatsoever.
• Email Clickers ... you have customers who click and don't purchase (bad), you have customers who click and buy stuff (the best), and you have the 90% of your email file who does nothing and needs to be experimented on.
• High Returners ... DO NOT MARKET TO THESE CUSTOMERS ANYMORE, OK??? It's horrible for your p&l ... do not send Email Campaigns and/or Print Campaigns to these customers.

There's a fourth customer type we're going to spend the next few weeks talking about.

That customer?

The "deflationary customer".

In a perfect business, you want an "inflationary customer". You know ... the kind that Apple possesses. The Apple customer will spend $1,000 on a phone ... while the "deflationary customer" will buy a $199 Android phone. Which customer delivers more gross margin dollars to your p&l?

So we'll spend a few weeks talking about deflationary customers.

Summer Schedule

Most summers I scale back publication from 5 days per week to 3(ish) ... so that you get a letter from me on Monday morning, Tuesday morning, and Thursday morning.

My regular schedule reboots after Labor Day.

By the way, not all retailers are in a death spiral (click here).