Analytics Sunday: Segmentation

The vast majority of folks run a Segmentation system.  Segmentation systems are, in large part, responsible for the analytics mess we're in today.  And yet, Segmentation systems are very effective, enabling us to make decisions that help our businesses grow.

Segmentation systems are popular because they are easy to implement.  Take Google Analytics ... it takes you all of a few minutes to put Google Analytics on your website, it costs nothing, and you're almost instantly obtaining actionable feedback.

Be aware that there are forces in your company that are not supportive of your Segmentation system.  Because Segmentation systems are easy to implement, folks prefer their own Segmentation system over your Segmentation system.  The bigger the company is, the greater the likelihood that somebody is crafting their own Segmentation system in opposition to your work.

I go back to Nordstrom.  My team essentially ran a Segmentation/Forecasting hybrid system.  This system was in contrast to many other teams.  When a business issue came up, at least six different teams tried to obtain answers, using their own systems.

• The Web Analytics team presented one-channel findings via Coremetrics.
• The Credit team presented credit-centric customer findings via their database, a database that you weren't allowed access to.
• The IT team had analysts that ran ad-hoc queries using SQL against the customer data warehouse, providing their own analytical viewpoint.
• The E-Mail marketing team provided their results, based on data from CheetahMail, results that spoke to the performance of e-mail marketing campaigns.
• Merchants used merchandising systems to report on the performance of their products.
• My team ran our analytics against the customer data warehouse, trying to cobble together a story of customer behavior across all channels (a "Multichannel Segmentation System, if you will).

Who was right?  Everybody, of course.  And nobody, of course.  You could only believe part of the story the Web Analytics team told, but their story was only valid for 10% of the business.  You could believe part of what the Credit team shared, but it was valid for 20% of the business.  You could believe some of what the IT team suggested, as long as the IT team audited their own queries properly.  You could believe some of what the e-mail marketing team said, but their results were only valid for 3% of the business.  You could believe some of what the merchants said, because their systems tied out to Finance, so that gave them some credibility.  And you could only believe some of what my team said, because we weren't responsible for any marketing activities, so our results, while unbiased, were not accepted because we didn't reside in various marketing silos.


Now, an outsider might look at this and say, "... whoa, why not just come up with one version of truth, you have a multi-channel customer database, just have Kevin's team share the results and that's it, nobody else works on the problem ... this is the problem with a silo-based approach to marketing and analytics."


Ha!


The problem is that Segmentation systems are easy to implement, so no matter what you do, it is human nature for folks to seek their own version of truth.  And trust me, folks will seek their own version of truth.


If you are going to rely upon a Segmentation system in your company, you better be ready for competition.  There will always be somebody out there who thinks they can do a better job than you can do ... that person is likely to reside in the IT department ... that person is likely to reside in a different business unit ... heck, I've even observed that person rising out of the finance department!


And you better be ready to be wrong.  The problem with running a Segmentation system that is easy to use is that folks in the Optimization camp, the Prediction camp, and the Forecasting camp all are ready to use advanced analytics to demonstrate that your data is "wrong".  You'll run a query that shows that "multichannel customers are the best customers", and somebody like me, running a Forecasting system, will blow your hypothesis out of the water.  Oh boy.


If you're going to run a Segmentation system, you almost have to focus your efforts on one word ... "profit".  Always be the person who knows how profitable every activity is/was.  The majority of your co-workers, folks running rogue Segmentation systems, are not going to go to the effort to calculate profit.  Profit gives you an advantage over everybody else, in fact, you instantly gain a valuable ally ... the finance department!!

Measuring Marketing Campaigns That Fail To Engage The Customer

We've all been there.

Somebody in the Marketing Department is charged with creating a campaign.  And oh boy, OH BOY, they absolutely hit one out of the park ... maybe they come up with some magical contest coupled with free shipping and 20% off in the middle of January, 2010.  For the next week, sales go BONKERS!

The marketer is celebrated.  She's nominated for a company award, and at the quarterly meeting, she's presented with $800 and a free plane ticket valid for travel anywhere in the continental forty-eight states.  Five weeks later, she's celebrating her genius at Hilton Head, while you're grinding through the analysis of one of an endless number of non-descript e-mail campaigns that are lucky to cause one in seven-hundred customers to purchase.


We're a short-term society.  We demand sales, NOW, and we reward people for what they've done lately.


Our businesses, however, thrive when we fertilize the customer file.


The truth is that many of our marketing campaigns fail to "engage" the customer.  In other words, when we measure the success of a marketing campaign, we take a "short-term" approach ... we simply ask if an e-mail marketing message or catalog or social media effort cause the customer to buy in the twelve, twenty-four, or forty-eight hours after the campaign.


Look at the image at the top of this post.


This is a "rolling-twelve-month" buyer analysis of the e-mail channel within a business.  We construct the query in parts.

• Count all customers who purchased via e-mail from 1/1/2009 to 1/1/2010.
• Then count all customers who purchased via e-mail from 2/1/2009 to 2/1/2010.
• Then count all customers who purchased via e-mail from 3/1/2009 to 3/1/2010.
• Repeat this process after every month, through current.

Plot the count of twelve-month buyers within the channel.  You end up with a graph similar to the one at the start of this post.


When a marketing campaign fails to "engage" the customer, you'll see a trend like the one illustrated in the image at the start of this post.  At month sixteen, the company ran a special e-mail marketing campaign, and customers LOVED IT ... twelve-month buyers increased by nearly 14,000.

Nine in ten Executives would say that the campaign "worked".


One in ten Executives would ask for a rolling-twelve-month buyer analysis a year later.  This rare set of Executives are rewarded with a very interesting finding!

Look at what happens in month twenty-eight ... the number of twelve-month buyers decrease by more than 12,000.


In other words, the vast majority of customers who purchased from the marketing campaign did not purchase again from e-mail, falling off the e-mail twelve-month buyer file.

Oh boy.


Now, granted, these customers may have purchased from other channels, so it's wise to run this analysis for your total buyer file, and for all micro-channels within your business.


This is common.  Marketers are rewarded for short-term efforts.  Our job is to analyze the long-term impact of our decisions.  Long-term, this campaign failed, because it harvested sales that did not translate into long-term business.

Forecast Forensics + Digital Profiles: Converting Factors

Here's where the rubber meets the road, as they say!

Yesterday, we created four factors.  Today, we assign customers to one of sixteen Digital Profiles.

WARNING:  Geeky math alert ... feel free to skim if you don't like math!

We score each customer, by standardizing each variable (remember, yesterday we calculated the mean and standard deviation of each variable on a twelve-month basis), then we multiply the coefficients in the Component Score Coefficient Matrix by the standardized variables.  Here's the Component Score Coefficient Matrix:

At this point, I have four factors.

Next, if a factor has a value greater than or equal to zero, we assign a value equal to one, otherwise zero.  Once we do this, we combine four factors by two values each, yielding sixteen Digital Profiles.  Here's my SPSS code, if you're interested (the four factors are f1, f2, f3, and f4).

compute d1 = -0.0000.
compute d2 = -0.0000.
compute d3 = -0.0000.
compute d4 = -0.0000.
compute dp =  00.
if (f1 ge d1) and (f2 ge d2) and (f3 ge d3) and (f4 ge d4)  dp = 01.
if (f1 ge d1) and (f2 ge d2) and (f3 ge d3) and (f4 lt d4)  dp = 02.
if (f1 ge d1) and (f2 ge d2) and (f3 lt d3) and (f4 ge d4)  dp = 03.
if (f1 ge d1) and (f2 ge d2) and (f3 lt d3) and (f4 lt d4)  dp = 04.
if (f1 ge d1) and (f2 lt d2) and (f3 ge d3) and (f4 ge d4)  dp = 05.
if (f1 ge d1) and (f2 lt d2) and (f3 ge d3) and (f4 lt d4)  dp = 06.
if (f1 ge d1) and (f2 lt d2) and (f3 lt d3) and (f4 ge d4)  dp = 07.
if (f1 ge d1) and (f2 lt d2) and (f3 lt d3) and (f4 lt d4)  dp = 08.
if (f1 lt d1) and (f2 ge d2) and (f3 ge d3) and (f4 ge d4)  dp = 09.
if (f1 lt d1) and (f2 ge d2) and (f3 ge d3) and (f4 lt d4)  dp = 10.
if (f1 lt d1) and (f2 ge d2) and (f3 lt d3) and (f4 ge d4)  dp = 11.
if (f1 lt d1) and (f2 ge d2) and (f3 lt d3) and (f4 lt d4)  dp = 12.
if (f1 lt d1) and (f2 lt d2) and (f3 ge d3) and (f4 ge d4)  dp = 13.
if (f1 lt d1) and (f2 lt d2) and (f3 ge d3) and (f4 lt d4)  dp = 14.
if (f1 lt d1) and (f2 lt d2) and (f3 lt d3) and (f4 ge d4)  dp = 15. 
if (f1 lt d1) and (f2 lt d2) and (f3 lt d3) and (f4 lt d4)  dp = 16.

With this logic ... and the instructions from above, I have sixteen Digital Profiles.

Next Week:  We describe each of the sixteen Digital Profiles that will be used in our Forecast Forensics analysis!  Contact me if you'd like to have your own customized Forecast Forensics / Digital Profiles analysis.

Forecast Forensics + Digital Profiles: Creation Via Factor Analysis

Here's the variables that I elected to enter into creation of sixteen Digital Profiles (contact me for your own customized project):

• Data for the past twelve months ... using the scoring algorithm from the past twelve months to score prior years as well.
• Frequency:  Orders in past year.
• Items per Order:  Total annual items (20) divided by total annual orders (4) = 5.00.
• Price per Item:  Total annual demand ($800) divided by total annual items (20) = $40.00.
• 1/0 Indicator:  Did customer buy using telephone channel in past year?  1 = yes, 0 = no.
• 1/0 Indicator:  Did customer buy using all other online channels in past year?
• 1/0 Indicator:  Did customer buy using last-click attribution to e-mail in past year?
• 1/0 Indicator:  Did customer buy using last-click attribution to search in past year?
• 1/0 Indicator:  Did customer buy using last-click attribution to social media in past year?
• 1/0 Indicator:  Did customer buy using last-click attribution to mobile in past year?

So, we create a dataset that has one year of data, with these attributes.

WARNING:  The rest of this post gets really "geeky" ... so if you don't like math, move along, there's nothing to see here!


Here are descriptive statistics for our variables:

The means and standard deviations are used later, when I want to create each of four factors.


Next, we run a factor analysis / principal components analysis, extracting four factors.  Here is the rotated component matrix:

In this analysis, we're looking for metrics with an absolute value greater than 0.20 ... this helps us identify the variables that contribute to each factor.

• Factor #1 = Frequency, Mobile, and Social.  This factor likes loyal customers who have migrated to mobile and social channels.
• Factor #2 = Telephone, Not Online.  In other words, this factor favors old-school shoppers who call the contact center to place an order.
• Factor #3 = Many Items per Order, Low Price per Item:  These customers like cheap items, and they buy lots of cheap items!
• Factor #4 = E-Mail + Search:  Customers who buy via e-mail and search, not necessarily other online channels, fall into this factor.  Kinda makes one wonder if e-mail causes search to happen, doesn't it?

Up Next:  We'll create sixteen Digital Profiles from the four factors extracted from this analysis.

Forecast Forensics + Digital Profiles: A Marriage Made In Heaven!

It's time to combine two fantastic methodologies, yielding a highly robust framework for understanding just what the heck is happening in your business!

Over the next several weeks, I will combine Forecast Forensics with Digital Profiles, illustrating how a multi-channel business has a customer base that his moving in many different directions, all at the same time!

Sound like the business you're managing?  Probably!


Tomorrow, we get started.  The database I'm using has several years of purchase history.  I will analyze six key channels:

• Telephone:  Customers ordering via the phone, primarily from catalogs.
• Online:  Pure online orders, minor online channels (affiliates), online orders driven by catalogs.
• E-Mail:  Orders with last click attributed to e-mail.
• Search:  Orders with last click attributed to search.
• Mobile:  Orders with last click attributed to mobile app or mobile website.
• Social:  Orders with last click attributed to social media.

I can hear some of you grumbling already, bemoaning the fact that I'm using last click attribution in this series.  Well, why don't you use this opportunity to take the methodology I'll share with you, and apply your attribution system to this methodology?  Better yet, why don't you freely publish the results so that everybody can benefit?


So, we'll take six channels, and we'll explore how customers are segmented (using Digital Profiles).  Then, we'll use the Forecast Forensics methodology to illustrate how the business is likely to evolve in the future, given what we've learned about customer behavior.


Forecast Forensics + Digital Profiles:  A marriage made in heaven!

Dear Catalog CEOs: Channel Trends

Dear Catalog CEOs:

One of the most interesting findings in my Catalog PhD projects (Print / Kindle) is the evolving nature of channels.

You remember the "multichannel era" ... i.e. 2000 - 2005 ... everything had to be linked together, integrated, in harmony, a direct response of the "wild west" internet of 1995 - 2000.


The evolution of the internet seems to have changed things.  It increasingly looks like 2005 - 2010 was a "sorting out" period ... customers found the channels that worked best for them.  In 2011, customers appear to be more "set" in their ways, as is evidenced by the Migration Probability Tables that I run.


Catalog / Telephone:  The 55+ rural audience really sorted themselves into this bucket.  They're comfortable with historical shopping habits, and aren't ready to do what pundits want them to do.  These folks party like it's 1999, if you will!  Be very careful if this is your primary source of new customers.


Catalog / Online:  This used to be a larger segment, but is increasingly fragmented.  A 45-54 year old suburban audience seems to skew to this choice.  This audience doesn't need 248 pages in the mailbox ... actively test sending 40% of your current page count to this audience, you're likely to get 90% of the demand on 40% of the pages.  Test, test, TEST!

Catalog / Retail:  Less effective than previously believed to be.  Even less effective among retail-only buyers.  Think about it ... the media has to motivate a customer to get in her car and drive to the store ... and that trip has to be unplanned ... that's HARD to do.  What would motivate you to get in your car to make an unplanned trip to J. Crew?  Be honest.


Online / E-Mail:  An increasingly dangerous combination of best customers and discount/promotion fanatics.  Reduce catalog pages to this audience ... once the shift migrates from catalogs to e-mail, you can significantly reduce page counts, and reduce frequency.  If you have a retail channel, then pay CLOSE attention to the ways that e-mail drives retail traffic, many retailers seriously undercount the impact of e-mail on retail.


Online / Search:  Search is surprisingly integrated with other channels, whether you know it or not.  Catalogs and e-mail cause searches to happen ... spending money on search is increasingly viewed as part of a two-step conversion process.


Online / Affiliates:  More incremental than previously believed to be, increasingly viewed as part of a two-step conversion process.  Surprisingly, customers who purchase via an Affiliate are unlikely to use an Affiliate on a subsequent purchase.

Sales Force:  I've yet to run across an instance where assistance from live human beings is anything but transformational, from a long-term value standpoint.  A relationship with an actual human being trumps any algorithmic channel-based experience.  Relationships with actual, live human beings represents all that social media ever aspired to accomplish.  Add humans to your business, as soon as possible!


Social:  Maybe the least impressive new channel ever created ... while at the same time being the most important channel since the advent of e-commerce.  Has almost no impact on annual spending behavior among a 50+ audience, has transformational impact on annual spending behavior among the 13-29 audience.  A channel buried in misinformation, hype, and mis-allocation of sales.  F-commerce (Facebook) could (or not) thrive as a sub channel (like e-mail thrives as a sub-channel within online).  Twitter ... not so much.  What we know is this ... what we see today won't be what we see tomorrow, everything will evolve and change.


Mobile:  Early returns suggest it will be transformational, in the way that e-commerce obliterated the traditional catalog marketing channel.  Early returns suggest that this transformation won't happen as fast as the pundits suggest it will.  Think of mobile as being in the Explorer/Netscape/Yahoo!/Hotmail/AOL phase that e-commerce was in back in 1998.  Generational differences are significant, critical to understanding how to leverage this new channel.  Mobile is likely to crush internal org structures, and is likely to cannibalize the living daylights out of the online channel.  Expect the mobile/online debate to rage like the catalog/online debate of 2000-2005.


Multi-channel:  Dead in the water.  Multi-channel customers do not spend more than single-channel customer.  Go back to 1995, and compare your annual retention rate and annual orders per buyer to comparable metrics in 2010 ... no difference (or worse than 1995), right?  Well, how is that possible?  I mean, you've added every channel under the sun, and your metrics have not fundamentally changed, right?  Adding channels does not translate to sales increases ... actually, adding channels might simply mitigate sales decreases.  Knowing demographics/lifestage is more critical, it dictates the effectiveness of channels.

Matchbacks:  Flawed.  Accounting for significant mailing waste.  Without a matchback, you're sunk.  With a matchback, you're sunk in the opposite direction.  Need something "in-between".  Catalog PhD sure helps!


Fragmentation:  This is what the promise of multi-channel became.  You'll have a veritable plethora of channels, and customers will self-select themselves into the channel combinations that satisfy their needs ... some will be anchored within a channel (a 63 year old New Hampshire woman anchored in the catalog/telephone channel), some will float between a half-dozen channels (a 32 year old suburban woman in Kansas City).  Our job is to segment (i.e. Digital Profiles), then to reduce/increase marketing expense and targeting strategy as appropriate.  Our job is not be everything to every customer, while at the same time offering many channels as appropriate ... a tough tightrope to walk.

Analytics Sunday: United Or Divided

We care about Analytics Systems, because by understanding where we stand, and where our Executive teams stand, we better understand why we succeed or fail.

Take this brief, two-question quiz.  You'll understand where you fall on the grid.  Answer how your Executive team might respond, and plot where they fall as well.

1. When deciding who should receive a marketing promotion, do you like to segment customers into like groups (1), or do you like to create models that predict how likely customers are to respond to marketing messages (9)?
2. When considering a new marketing strategy, do you like to run a series of simulations that predict how customers might behave in the future (1), or do you like to run a series of tests that reveal an optimal marketing strategy to employ against your customer base (9)?

Plot the answer to the first question on the following chart ... (1) on the left, (5) in the center, (9) on the right.

Plot the answer to the second question on the following chart ... (1) on the bottom, (5) in the center, (9) on the top.

If you and your Executive team aren't plotted near each other, you've got the potential for communication problems.

Analytics Systems matter, they represent the difference in philosophy that divides or unites us.


Here's an old-school example:  Catalog marketers often employed a Segmentation/Forecasting hybrid model to measuring advertising effectiveness.  For instance, Lands' End used a segmentation strategy with "current" customers, followed by 0-3 month recency, 4-6 month recency, 7-9 month recency, 10-12 month recency, and beyond in three month increments.  Eddie Bauer and Spiegel used a similar strategy ... "current" customers, followed by 0-6 month, 7-12 month, and beyond.  This segmentation strategy served dual purposes.  First, catalog performance was measured within each segment, and that was important (part of the Segmentation system).  Just as important was a subtle twist incorporated within a Forecasting system.  Customers were locked-into the segment at the start of a quarter/season, and only moved out of the segment into "current" quarter/season status if the customer purchased.

• January 1 = 0 Current Season Customers, 100 0-3 Month Buyers.
• June 30 = 67 0-3 Month Buyers.
• Rebuy Rate For 0-3 Month Buyers = 1 - (67/100) = 33%.

This was a subtle, but very clever implementation of a Segmentation/Forecasting hybrid system.  The analyst knew how customers performed in each catalog (Segmentation --- understanding marketing campaign effectiveness), and just as important, the analyst was able to understand overall repurchase dynamics independent of marketing activities, and understand file strength implications for upcoming quarters/seasons (Forecasting --- understanding and forecasting customer performance independent of marketing campaigns). 


Back in 2001, I was trying to implement an Optimization/Prediction hybrid system, so I abandoned this type of segmentation scheme at Nordstrom ... the decision did not sit well with a faction of individuals who were loyal to the Segmentation/Forecasting hybrid system mentioned above.  Heck, it was as if I just obliterated all analysis reporting in the company!!!  My decision was not popular, at all ... when you and Management are far apart on the systems grid, it doesn't matter how bright you are, you're in big trouble.


Web Analysts frequently run into this reality, just read their tweets, calling Executives "HiPPOs", meaning "Highest Paid Person's Opinion".  You see, the Web Analyst often uses a Segmentation system that is hard-wired into the software solution used by the Web Analyst.  The person the Web Analyst is trying to convince frequently uses a Forecasting system ... take a merchant, who only cares if merchandise sells, pure and simple.  This person isn't evaluated on the performance of an individual catalog, this person is evaluated on customers loving her merchandise on an annual basis.  These are two individuals, running two different systems ... the merchant wants to know what is going to work in the future, the Web Analyst cares about what happened to conversion rates last month.  When individuals run different systems, disagreements happen.


It's really important to know where you stand on the Analytics System continuum.  It's just as important to understand where your Executive team stands on the continuum.  If you're in a similar place, you are united.  If you're in different places, you may be divided against each other.