Criticism of Comp Segment

As mentioned a few days ago, there are criticisms of my Comp Segment framework. I've heard 'em since I began sharing the methodology widely ... back in 2012/2013.

Let's hear some of the criticisms/questions:

You only control for purchase frequency. Shouldn't you also control for dollar amount? Why control for 2 orders when some place two orders for $50 each and others place two orders for $125 each?

• Yes, controlling for dollars is even better.
• The problem you face is that smaller companies have fewer customers, so as you add dollar constraints you reduce the size of the audience and therefore increase noise, making inferences more challenging.

Why two purchases in the past year? Why not three? Why not eleven?

• There is no right/wrong answer to this question.
• You want to avoid infrequent shoppers as they're not representative, you want to avoid first-time buyers where possible (as they're influenced by marketing channel used during acquisition), and you want to avoid loyal buyers who are highly influenced by marketing.

Shouldn't large retail brands like Macy's use different criteria than two purchases in the past year?

• Yes, and Macy's has enough customers to eliminate noise.
• Control for both purchase frequency and dollars spent, and if the brand has a loyal customer base, you can move up to 3/4/5ish purchases in the past year.

Won't I get different results if I look at different segments of customers?

• Quite possibly, yes.
• But the "direction" of the results, unless you have a very noisy customer base, will be comparable. And that's what is important.

How do you "know" that the issues identified by the analysis are merchandising issues?

• You analyze comp segment performance for new items, and for existing items. You analyze by merchandise division. If you see differences, you have merchandising issues. If every analysis looks identical, then positives/negatives are marketing driven.

Shouldn't you use machine learning or AI to measure the impact of merchandise productivity, controlling for a myriad of issues?

• Absolutely! Have at it.
• However, if you do 2% of the work (that's how easy this is) you'll get 95% of the answer. For just 2% of the work. Who wouldn't do 2% of the work to get 95% of the answer?

Shouldn't I average results across many different segment?

• Go ahead, do that. Just be sure to not let marketing issues cloud your results.
• Of course, one wonders why you haven't already been doing that for the past decade?

There's nothing you can say to convince me that I should be doing this work. Your work is just plain wrong.

• Then don't do it. Do your work your way. That's fine!