Putting Together the Puzzle

Yesterday I shared a high level sales plan, one where we figured out that our business would generate $28.2 million in sales and $2.6 million in profit.

Is the plan something we can take to the bank?

Well, you have the "bottom's up" crowd that estimates a business plan based on individual segment performance in marketing campaigns. It's a lot of work, and it isn't any more accurate than doing the simple calculation I performed yesterday. I know this because I spent a decade building plans at Eddie Bauer & Nordstrom and have the scars to prove my point.

The plan, of course, is subject to a lot of variability.

For instance, over the past five years, annual rebuy rates have varied as follows for this brand:

• 36%.
• 34%.
• 38%.
• 35%.
• 37%.

In Excel, it's easy to calculate the mean & standard deviation:

• Mean = 36%.
• Standard Deviation = 0.016.

If results are normally distributed (and they usually are), this tells us something about the variability around rebuy rates.

That's the outcome of 10,000 simulations of rebuy rate, given how rebuy rates have varied over the past five years.

We perform the same exercise for other key metrics ... rebuy rate, spend per repurchaser, new + reactivated buyers, spend per new/reactivated buyer.

For each of 10,000 simulated outcomes, we calculate projected net sales, and we calculate projected profit. Here is the histogram for projected net sales.

Oh oh.

Based on a normal range of results from prior years, we observe quite a range of possible outcomes, don't we?

• 10% of the time, net sales will be under $26.5 million.
• 25% of the time, net sales will be under $27.4 million.
• 50% of the time, net sales will be +/- $28.5 million.
• 25% of the time, net sales will be above $29.6 million.
• 10% of the time, net sales will be above $30.6 million.

Honestly, in our business, that's not a lot of variability.

But for your CFO, that is WAY TOO MUCH VARIABILITY.

Now imagine having to layer in the impact of COVID on top of those numbers.

Tomorrow I'll share with you what profit looks like for our business.

Is My Plan Risky?

You start with a high-level plan for the upcoming year, based on actual data from the year prior. This is a common-sense approach to planning, one that so darn many companies use.

12 Month Buyers:

• 200,000 buyers, 36% repurchase rate, $160 spend per repurchaser.
• 200,000 * 0.36 * 160 = $11,520,000 sales next year.

New + Reactivated Buyers:

• 130,000 New/Reactivated buyers, $128 spend per purchaser.
• 130,000 * 128 = $16,640,000 sales next year.

Total Volume:

• $11,520,000 + $16,640,000 = $28,160,000 sales next year.

Other Metrics:

• 40% of sales flow-through to profit.
• Expected Ad Cost expenditure = $5,632,000.
• Expected Fixed Costs = $3,000,000.

Expected Earnings Before Taxes Next Year:

• $28,160,000 * 0.40 - $5,632,000 - $3,000,000 = $2,632,000.
• 2,632,000 / 28,160,000 = 9.3% of Net Sales.

Is your plan "risky"?

In other words, you are signing up for $2,632,000 profit on net sales of $28,160,000. How likely is it that your "promise" will happen? Could you end up with a lot more in sales? A lot less? And how would that impact profit?

Tomorrow I'll show you how variable forecast "could" be.

Communicating Risk

Remember yesterday, when I shared that something you pencil out in a spreadsheet as a "good decision" is actually a decision that has risk? In my example, what appeared to be a profitable decision actually had a 23% chance of being unprofitable.

Here's the biggest reason people don't evaluate risk. It is "risky" to communicate risk.

I worked for a person who took a few stat classes in college and therefore thought he was a brilliant statistician. When I communicated risk to him, he got uncomfortable, then he got angry, then he misinterpreted what he learned in his three stat courses, then he said things that were completely wrong but said them as if he were right so anybody in his orbit took his side, causing everybody in the room to be wrong.

What did he misunderstand about risk?

He believed that the "average" outcome was a "certain" outcome.

And if actual results came in "below average", well, that was your fault.

And if actual results came in "above average", well, that was because the merchant was brilliant.

I learned to NEVER communicate a probability to this person, because this person was incapable of understanding probabilities. He'd have been a terrible Texas Hold 'em player and a really sore loser if he had to deal with a "bad beat" on the river.

Instead, I sandbagged everything.

I never submitted a business plan assuming an "average" outcome. I'd calculate the probability of a less-than-optimal outcomes, then submit a less-than-optimal outcome. Business would then "hopefully" perform at an average level, in which case my plan would be exceeded, and everybody would be happy ... including my boss who didn't (couldn't) understand probability and risk.

It's awfully hard to properly communicate risk to individuals who struggle with assessing risk. Come up with a strategy that mitigates the downside while promoting an upside that makes people happy.

Decisions and Risk

You have a paid search program, and you expect the program to deliver the following metrics this month:

• Conversion Rate of 1.5%.
• Average Order Value of $90.
• Average Cost per Click = $0.50.
• Profit Flow-Through = 40%.
• Expected Profit = 0.015 * 90 * 0.40 - $0.50 = $0.04.

You tell your agency that all efforts need to break-even. The data suggests that given the metrics, you should make a few pennies per click, on average.

But how much "risk" is there in that situation?

Here's the result of a 10,000 simulated runs, runs where I apply variability surrounding the conversion rate.

On average, you are generating $0.04 of profit per click ... but the amount of profit could vary between a loss of $0.10 and a profit of $0.17 in most cases. In fact, you'll lose money 23% of the time. In other words, even though your plan suggests that you should make money, 23% of the time you will not make money.

If you were told that your decisions needed to be profitable, would you make a decision that will lose money 23% of the time?

Almost every reader out here fails to perform this style of analysis.

But every reader out here SHOULD perform this style of analysis. All decisions have some sort of risk tied to 'em. At minimum, you need to understand how much risk surrounds your decision.

Testing Budget

Earlier in the week, I mentioned that I used to negotiate with the CEO of the Online Division at Nordstrom for my testing budget.

In other words, we had a discussion each year, and we pre-agreed upon the sales/profit impact of my tests.

I recall in 2003 that the budget was set at 3% of sales. For a $350,000,000 division, this was roughly $10,000,000 in sales and $3,000,000 profit. The CEO was willing to lose $3,000,000 profit because what was learned in executing the tests was worth $3,000,000 profit.

If you do your job as a Virtual Chief Performance Officer properly, you'll generate enough downstream profit to more than make up what you lose executing the tests.

Do you have a testing budget?

Odds are 90% of you don't have a testing budget, and that says something about your historical ability to prove that what you learn from testing has long-term value.

For instance, the testing budget at Nordstrom was what we used to determine that catalogs (via holdout tests) generated no incremental profit to the company ... which allowed us to allocate $36,000,000 of ad-cost to other activities ... half to paid search ... which generated a profit ... which caused our online sales to increase when we pulled all that paper out of the ecosystem. The testing budget (where we were willing to lose $3,000,000 profit per year to learn) allowed us to learn enough to make tens of millions of dollars of profit.

That's what you can accomplish as a Virtual Chief Performance Officer ... implementing a simple testing budget and a credible testing plan and a communication plan for the results of your tests.

-188 + 192*x

It was 1991 at Lands' End. We were greatly ramping-up our testing work. And when we wanted to execute a test, we needed to understand how the results might "vary".

Back then, a test was sampled from the population who would receive a catalog. Maybe that audience was 4,000,000 customers. If the catalog was a productive catalog, it might generate $10.00 per catalog mailed. If the catalog wasn't productive, circulation would be reduced and the catalog might generate $4.00 per catalog mailed.

If you want to measure a 10% difference in sales for two groups performing around $4.00 per book, you need fewer customers than if you are trying to measure at 10% difference in sales for two groups performing around $10.00 per book. This is an issue called "heteroscedasticity".

So I built an equation that measured variability around different dollar-per-book estimates. The equation was a simple one:

• -188 + 192*(Expected Dollar per Book).

If we expected one group to generate $4.00 per book and the control group to generate $3.60 per book, we'd calculate the variability at point estimate:

• $4.00 = -188 + $192*4.00 = 580.
• $3.60 = -188 + $192*3.60 = 503.

Then we'd enter the data into our statistical equation.

• (4.00 - 3.60) / SQRT(580/25000 + 503/25000).
• T = 1.92.

As long as T > 2.00, we would execute the test with the sample size promoted by the equation.

In this case, the sample size was too small, so we had to increase it.

• (4.00 - 3.60) / SQRT(580/30000 + 503/30000).
• T = 2.11.

You probably already have a calculator that you enjoy using. If not, contact me and we'll get something set up for you for your data at minimal cost (kevinh@minethatdata.com).

But I Cannot Afford THAT Many Customers

This is the part of the discussion where the Virtual Chief Performance Officer hears the grumbles, from near and far.

• "We cannot afford to test THAT many customers!"

You can't afford NOT to test THAT many customers.

When I worked at Nordstrom, the Online CEO and I had to negotiate the size of the "testing budget". I needed a lot of customers, he needed a lot of sales, and he needed to learn things. So compromises had to be met.

I'd pull out a graph that looked something like this (don't worry about the y-axis at this time ... worry about where it is high vs. where it is low).

If we only sampled 10% of the audience, we had a lot of sampling error ... and that meant we'd never be certain we had the "right" answer.

If we sampled 40% of the audience, we had minimal sampling error ... and that meant we'd likely have the right answer but would give up a lot of sales to learn the right answer.

So we'd agree that "x" percent of the population was appropriate for how much we wanted to learn vs. the amount of sales/profit we were willing to pay to learn the information.

That's what we're talking about:

• How much profit are you willing to pay in order to learn something about your business?

For too many of you, the answer is $0. You're not willing to pay anything to learn.

And as a result, you don't learn anything.

It costs money to learn something. You already know this, you attended college, right? So as a 20 year old, you were willing to pay something to learn. But as a seasoned 40 year old Professional, you're not willing to pay anything to learn. Hmmmmmm.

Be willing to pay something to learn something.