Analytics Sunday: Forecasting

I get a lot of questions about this one.

• "What the heck is the Foreasting system?"
• "How is it different than other systems?"
• "Why is this the system you, Kevin, gravitate to?"

Every individual, knowingly or unknowingly, runs an Analytics system.  Your system, of course, represents the basis for how you approach obtaining answers to questions.

Segmentation, Optimization, and Prediction systems are predicated on finding answers to marketing questions.

The Forecasting system is predicated on finding answers to customer questions.

Do you understand the distinction?

Here's the reason I like to run a Forecasting system.  I'm working with a client last year.  The client is frustrated ... a marketing campaign in August worked great, a marketing campaign in September worked terribly.  The question from the client ... "how do we get both campaigns to work perfectly?"


In a Segmentation system, and Optimization system, and a Prediction system, you go to work, analyzing the differences between each marketing campaign.  You compare and contrast, you do a ton of work, you highlight the differences, and you work on making sure that next year, things work out better.


In a Forecasting system, you ignore the campaigns.  I know, that goes against every single thing you've been trained to do.


Here's what I did.  I froze the customer file as of 8/1.  I then measured repurchase rates from 8/1 to 9/30.  I analyzed reactivation rates among 13+ month customers.  I analyzed new customer acquisition in this sixty day period.  I compared these metrics across the past three years.


Guess what?  Repurchase rates were the same as last year.  Reactivation rates were the same as last year.  New customer acquisition counts were the same as last year.  All three metrics were better than two years ago, and better than three years ago.


In other words, the overall health of the business was fine.  Marketing campaign performance was variable.


This is a distinction that wastes a tremendous amount of time withing companies.  Under the Segmentation, Prediction, or Optimization systems, companies analyze campaigns that are inherently unstable, unpredictable, unreliable, and variable.  Significant effort is made to improve campaigns that, on average, have a +/- 30% variability rate without or without improvement.  In other words, you could have a campaign that performed 50% above plan last year, you improve the campaign by 10%, and the campaign performs at -15% to the prior year ... this isn't because the campaign performed bad, it is because of random variability.


When you measure performance over time, independent of campaign performance, you often see a different story.  You often observe consistent trends, a stable file, a file that is driven not by marketing campaigns, but by organic purchases and new customer increases.


This is the magic of the Forecasting system.  You focus efforts on pre/post periods, you ignore marketing campaigns, you extrapolate current trends into the future to see where your business is heading.


Now, I can hear the complaints, from near and from far.

• "If you figure out how to make both marketing campaigns work, your business will be twice as good, so you're better off fixing the one that doesn't work!"

I wish the world worked like this.  When viewed through the eyes of the Segmentation, Optimization, or Prediction systems, this is possible.


When viewed through the Forecast system, you realize that your customers are going to repurchase at 45% rates, purchase two times per year, and buy three items per order.  These rates, on an annual basis, seldom vary by more than +/- 10%.  

You miss the reality of your business when you focus on marketing campaigns.


That's why I have shifted, over time, from a Optimization/Prediction system, to a Forecasting system that skews a bit toward Segmentation.  I can more quickly focus an Executive team on the real reasons why a business is succeeding or failing.

Analytics Sunday: Segmentation

The vast majority of folks run a Segmentation system.  Segmentation systems are, in large part, responsible for the analytics mess we're in today.  And yet, Segmentation systems are very effective, enabling us to make decisions that help our businesses grow.

Segmentation systems are popular because they are easy to implement.  Take Google Analytics ... it takes you all of a few minutes to put Google Analytics on your website, it costs nothing, and you're almost instantly obtaining actionable feedback.

Be aware that there are forces in your company that are not supportive of your Segmentation system.  Because Segmentation systems are easy to implement, folks prefer their own Segmentation system over your Segmentation system.  The bigger the company is, the greater the likelihood that somebody is crafting their own Segmentation system in opposition to your work.

I go back to Nordstrom.  My team essentially ran a Segmentation/Forecasting hybrid system.  This system was in contrast to many other teams.  When a business issue came up, at least six different teams tried to obtain answers, using their own systems.

• The Web Analytics team presented one-channel findings via Coremetrics.
• The Credit team presented credit-centric customer findings via their database, a database that you weren't allowed access to.
• The IT team had analysts that ran ad-hoc queries using SQL against the customer data warehouse, providing their own analytical viewpoint.
• The E-Mail marketing team provided their results, based on data from CheetahMail, results that spoke to the performance of e-mail marketing campaigns.
• Merchants used merchandising systems to report on the performance of their products.
• My team ran our analytics against the customer data warehouse, trying to cobble together a story of customer behavior across all channels (a "Multichannel Segmentation System, if you will).

Who was right?  Everybody, of course.  And nobody, of course.  You could only believe part of the story the Web Analytics team told, but their story was only valid for 10% of the business.  You could believe part of what the Credit team shared, but it was valid for 20% of the business.  You could believe some of what the IT team suggested, as long as the IT team audited their own queries properly.  You could believe some of what the e-mail marketing team said, but their results were only valid for 3% of the business.  You could believe some of what the merchants said, because their systems tied out to Finance, so that gave them some credibility.  And you could only believe some of what my team said, because we weren't responsible for any marketing activities, so our results, while unbiased, were not accepted because we didn't reside in various marketing silos.


Now, an outsider might look at this and say, "... whoa, why not just come up with one version of truth, you have a multi-channel customer database, just have Kevin's team share the results and that's it, nobody else works on the problem ... this is the problem with a silo-based approach to marketing and analytics."


Ha!


The problem is that Segmentation systems are easy to implement, so no matter what you do, it is human nature for folks to seek their own version of truth.  And trust me, folks will seek their own version of truth.


If you are going to rely upon a Segmentation system in your company, you better be ready for competition.  There will always be somebody out there who thinks they can do a better job than you can do ... that person is likely to reside in the IT department ... that person is likely to reside in a different business unit ... heck, I've even observed that person rising out of the finance department!


And you better be ready to be wrong.  The problem with running a Segmentation system that is easy to use is that folks in the Optimization camp, the Prediction camp, and the Forecasting camp all are ready to use advanced analytics to demonstrate that your data is "wrong".  You'll run a query that shows that "multichannel customers are the best customers", and somebody like me, running a Forecasting system, will blow your hypothesis out of the water.  Oh boy.


If you're going to run a Segmentation system, you almost have to focus your efforts on one word ... "profit".  Always be the person who knows how profitable every activity is/was.  The majority of your co-workers, folks running rogue Segmentation systems, are not going to go to the effort to calculate profit.  Profit gives you an advantage over everybody else, in fact, you instantly gain a valuable ally ... the finance department!!