Lots of Variability

Yesterday I showed you what happens when you sample 25% of customers from a full population with a known outcome ($14.42 per customer, average) ... you end up with noise.

That's what happens when you aren't dealing with the entire population.

What happens when you deal with an even smaller fraction of the original population? Look at the x-axis and compare the values in the histogram to the image above.

Oh oh.

Notice that the overall average is "similar" ($14.32 sampling 5% of customers vs. $14.39 sampling 25% of customers vs. sampling 100% of customers and getting $14.42).

But the spread is huge!

Huge!

Some of the outcomes are around $13.75 ... and you CANNOT know if that outcome is the real outcome or a blip due to sampling variability. One of the outcomes was > $15.50 and you cannot know if that outcome is the real outcome or a blip due to sampling variability.

Look at the standard deviation metric ... 0.412 ... so much higher than the 0.113 we saw when sampling 25% of the original population. It means that your results will vary by +/- $0.82 instead of +/- $0.23.

The Virtual Chief Performance Officer has a job ... and that job is to STEER YOU AWAY from situations where you are measuring something with +/- $0.82 of variability ... steering you toward outcomes that are within +/- $0.23 of expectations.

Here's the problem we all face. We're told we're supposed to TEST outcomes, aren't we? You are supposed to be "data-driven".

But if you are "data-driven" and you make one little mistake ... testing without enough cases to test properly, well, you provide your company with GARBAGE, don't you?

This sin happens repeatedly in business. Smart data-centric folks making bad choices, not understanding variability, creating chaos in a company and then defending the chaos.

The Virtual Chief Performance Officer steps the company away from these situations. We'll talk more about these situations tomorrow.

Variability

One of the jobs of a Virtual Chief Performance Officer is to teach the concept of "variability" to employees ... and especially Executives.

In business, things don't happen on a predictable basis. There is always noise. ALWAYS! The real world is noisy too ... look at polling errors (again) associated with the Election.

Allow me to show you an example. Here is performance for twelve-month buyers in the next month. On average, customers spent $14.42 in the next month.

• Most spent $0.
• Some spent $50.
• Some spent $100.
• Some spent $500.
• Average = $14.42.

Let's conduct a little experiment here. I ran 30 samples from this audience. I randomly sampled 25% of the customers in this audience, measured the average spend in the next month, and then plotted a histogram of the distribution. Here is the histogram.

Most of the results are "similar" to an average of $14.42 ... but aren't exactly $14.42, are they? This is the definition of "sampling error". We sample from the original population, but the results don't end up EXACTLY like the original population. We have differences.

A Professional has to be able to deal with these differences. You will never know if the actual result ($14.42) actually happens. Sometimes you take a sample and you get $14.20 ... it's technically the same as $14.42 because it is drawn from the same population, but the observed outcome IS NOT THE SAME and you cannot ever know that it is not the same.

In this case the standard deviation is 0.113 ... so you can expect most of your results to vary by between +/- $0.23. If you get a result of $14.20 ... it might actually be $13.97 or it might actually be $14.43 (remember, the actual observed outcome was $14.42) and you don't know what direction your estimate is off by.

Tomorrow I'll show you what happens when you only sample 5% of the population instead of 25%.

A Side Effect of the COVID-bump

Is this what your COVID-bump looks like, when measured via a Comp Segment analysis?

If so, you've probably performed some advanced analytics (or hired somebody like me to perform them), and you may have observed an interesting side-effect of the COVID-bump.

• Sales of items selling at/above their historical average price are increasing dramatically.
• Sales of items selling below their historical average price are flat.

This is a fabulous outcome, of course ... it means that your sales just flow-through to profit at a very high rate.

A COVID-bump provides pricing integrity. Ask folks who produced toilet paper in Spring about pricing integrity. Pricing integrity leads to long-term profitability. Take advantage of the pricing integrity you may have earned via a COVID-bump, and you may have a more successful 2021-2022.

Trust

For the third of the audience who reads my content and tends to be catalog-centric, you have two delivery-centric vendors who will need to re-establish trust with you going forward. 

• USPS.
• FedEx.

One pushes your marketing to the customer.

One delivers your merchandise to the customer.

Both are now an obstacle to your success.

FedEx (you've told me this) spent months telling you to sub-optimize your business so THEY can optimize their business. Here is a recent story (click here).

And if the USPS is stopping delivery of some mail-in votes (right or wrong), how do they convince you that they ever ... ever ... actually did what was right for your business?

Institutions eventually break. When they break, something is lost ... then something new is created. You get to decide what is created. Get busy creating, ok?

New Customers Don't Know That Old Merchandise Is Old Merchandise

This fact keeps popping up, folks.

If you have a COVID-bump ...

... then you need to dig into the stuff that is driving your COVID-bump (click here for your customized project).

Like existing merchandise. A lot of the customers driven by the COVID-bump are coming from search. Within search, Google sure seems to like pushing prospects toward existing items, long-term winners that Google has a significant history "analyzing". When I look at what COVID-bump customers are buying, it's the stuff that has always worked. And why not? New customers don't know that existing merchandise is the stuff that has always worked. So if it has always worked, the numbers "optimize" and you end up preserving your assortment one cycle longer than you expected to preserve it.

Be careful analyzing new merchandise during the COVID-era. What works in October / November 2020 "could" work next year ... but we don't really know that, do we? Your experience, knowledge, and imagination are going to go a long way toward projecting a future assortment.

And for some of you, the fact that new customers don't know that old merchandise is old merchandise is an important finding that has practical (and profitable) ramifications for your future.

An Interesting COVID-Based Trend

We know what a COVID-bump looks like, via Comp Segment analytics.

We also talk frequently about the "organic percentage" ... the share of orders that happen without the aid of marketing.

Here was a trend for one company:

• November 1, 2018 = 55%.
• May 1, 2019 = 58%.
• November 1, 2019 = 62%.
• March 1, 2020 = 64%.

So this brand is getting healthier by the minute .... orders are consistently happening without the aid of marketing, and at increasing rates.

Then we look within the COVID-bump:

• April 1, 2020 = 63%.
• May 1, 2020 = 64%.
• June 1, 2020 = 61%.
• July 1, 2020 = 59%.
• August 1, 2020 = 58%.
• September 1, 2020 = 58%.
• October 1, 2020 = 57%.

This is what a SMART MARKETING DEPARTMENT looks like.

Why would I say that?

Well, it is obvious that the marketers figured out in April/May that "something" was going on. They read the Comp Segment tea leaves properly, and they put their foot on the marketing gas pedal, spending money to amplify a highly positive trend.

You leverage your business experience to understand what is happening behind the scenes. Behind the scenes at this company, somebody was trying very hard to amplify a positive trend. These are the kind of clients (or companies) you want to work with. They're trying hard to make things better.

What Does A COVID-Bump Look Like?

It's not hard to see it ... this is what it looks like across my client work, using my Comp Segment framework (you know how to calculate Comp Segment performance, right?):

If "normal" is +3%, we'll set the axis at +5%. Then in March we see a +5% comp, followed by a nice comp in April and a bonkers comp in May. From there, comps slowly begin to reset closer to normal. For the April - September timeframe, the average comp is +27%. That's what a COVID-bump looks like. Normal performance (+3%) becomes +27%, and the bump has a natural peak and then cools off.

Of course, if the maskless Midwest continues to burn down, then there may be additional restrictions, and those restrictions might inspire another COVID-bump. So we don't know what is going to happen. We do know we can use our Comp Segment framework to measure what is happening, and act accordingly.