Make Sure You Control For Key Attributes

When you control for recency, frequency, monetary value, channel, merchandise preference, and price ... you'll learn that certain attributes that appear important at an "indicator level" are not important at all, or are important for a very short period of time ... or are still very important. Anything is possible. But you have to control for key factors first.

Let me tell you a story about a client ... this client wanted me to prove that their loyalty program was "working". It wasn't working ... loyal customers in the loyalty program were spending 10% less per year and loyal customers not in the loyalty program were spending 10% less per year and just about every customer segment was off 10% year-over-year. The business just wasn't resonating with customers, and the loyalty program wasn't helping.

So that's what I presented.

Guess what?

The loyalty vendor didn't like what I shared. Go figure!!

So the Loyalty vendor puts together a slide that has a horribly biased bar chart ... the slide shows that customers with a Loyalty Indicator = Yes were 160% (or whatever the percentage was, I don't recall the exact number) more valuable than everybody else and therefore the Loyalty program was generating enormous value. The slide used words like "PROOF" and "VALUE" in all caps, so of course you had no choice but to believe the vendor ... a vendor with a vested interest in protecting the program defended by the "indicator variable".

I ran about 50 separate queries. 

I ran a query segmenting Mens customers from all other customers. Guess what? Mens buyers were more valuable.

I ran a query segmenting Online visitors from all other customers. Guess what? Online visitors were more valuable.

I ran a query segmenting discount/promo buyers from everybody else. Guess what? These buyers were more valuable.

I ran a query segmenting liquidations buyers from everybody else. Guess what? These buyers were more valuable.

In fact ... in all cases (for that client) the chosen attribute suggested customers were more valuable if they aligned with the chosen attribute.

Why did this happen? Because of the highly biased nature of the queries that some vendors and some marketers construct to prove their point. 

• The attribute isn't what matters.
• The fact that good customers do many things means that you can select nearly any attribute and many / most good customers will possess that attribute and therefore the attribute will show that customers are better.

Don't believe me?

Run the queries for yourself on your own data.

Seriously. Go. Now!!

Meanwhile, there are times when indicator variables are truly important. In a recent analysis I saw that an email subscriber variable (1 = yes, 0 = no) showed a 45% lift in a segmentation analysis ... but get this ... it showed a 52% lift in a logistic regression after controlling for recency and frequency.

In other words, I'm asking you to do some work ... be a bit more sophisticated than normal. Your indicator variable might well be meaningful and important. Prove it by controlling for other factors, ok?