June 26, 2007

Explaining The Matchback Mistake

Now that I've frustrated many of you by not aligning with catalog industry best practices (i.e. the right way to implement results from a "matchback" analysis), allow me to explain the philosophical issues surrounding the methodology.

Catalogers like to look at a "segment" of customers, folks with similar behavior, folks with consistent future performance.

For instance, assume it costs a cataloger one dollar to mail a catalog. Also assume that thirty-five percent of all demand flows-through the p&l, resulting in "contribution" or "variable operating profit".

We mail a catalog to this segment of 10,000 customers, folks who last purchased within the past three months, and have spent $250 - $499 in their lifetime with the company.

By measuring source codes, we learn that this segment spent $2.00 per customer over the telephone. We run a profit and loss statement, and observe the following:

Households 10,000

Demand $20,000
Flow-Through $7,000
Book Cost $10,000
Contribution ($3,000)

In other words, we lost money mailing this segment of customers.

This is where the matchback analysis comes in. Savvy catalog marketers partnered with list processing and compiled list vendors to "match" all customers who received a catalog, but ordered online instead, "back" to the catalog mailed to the customer. Typically, the most recent catalog mailed gets credit (and we can address all the flaws with that methodology another day).

In this instance, the "matchback" analysis shows that customers mailed this catalog also spent $2.00 online during the life of this catalog. This changes the profit and loss statement, illustrated below:

Households 10,000

Demand $40,000
Flow-Through $14,000
Book Cost $10,000
Contribution $4,000

Now all is good in the world! The catalog drove online volume, the profit and loss statement works. Catalog list processing vendors, compiled list vendors, paper vendors, and list rental vendors rejoiced because the catalog becomes a viable marketing vehicle responsible for the majority of the online volume harvested by a business.

This strategy works well when the online channel is incapable of generating its own volume. In 2007, this is often an incorrect and dangerous assumption. This is where mail/holdout testing comes into play.

Simply put, mail/holdout testing shows you how much online volume occurs if a catalog isn't mailed. The methodology points out the fundamental flaw in a matchback analysis.

For many catalogers (certainly not all, maybe not even half), half of the online demand will happen anyway if a catalog is not mailed. In these instances, the mail/holdout testing clearly illustrate this finding (see the last article, business model number three).

In the case of our profit and loss statement, adding in one dollar per customer instead of two dollars per customer changes the profit and loss statement, illustrated below:

Households 10,000

Demand $30,000
Flow-Through $10,500
Book Cost $10,000
Contribution $500

In this case, the segment is above break-even, so depending upon your profitability criteria, the segment can be mailed next year.

It is this last profit and loss statement that catalogers need to be evaluating.

Almost all catalogers are mailing too many catalogs due to flaws in the implementation of the matchback analysis. This isn't the fault of your list processing or compiled list vendor. It is our fault, we failed to adequately understand customer behavior.

At Nordstrom, when we killed our catalog division, our online division actually continued to grow sales, year-over-year. Matchback analysis suggested that killing the catalog would create a catastrophe. Our inventory management team nearly fainted, thinking the implosion would be epic!

Mail/holdout testing accurately forecast a subtle sales hit that would largely be offset by organic growth in the online channel. Within a month of killing the catalog, we observed that mail/holdout testing was right, that matchback analyses were highly flawed.

Another flaw in the implementation of matchback analysis is attributing online orders to the original source (which in most cases, is catalog).

In other words, the catalog marketer gives the online channel credit for taking the order, but says that the order could never have happened had catalog marketing not been responsible for originally acquiring the customer. This analytical technique assures that catalogs will always gain too much credit --- in these cases, I've seen orders generated by paid or natural search (i.e. Google) attributed to catalogs, because the customer was acquired via a catalog twelve years earlier. I'd stay away from this popular method of attribution.

I realize what I am saying is utter heresy to most in the catalog industry, as evidenced by the feedback I receive from you! As leaders, we have a responsibility to maximize sales and profit in the business models we support. Let's measure the evolution of our business in a fair manner. We need to take our catalog silo hat off, and put our brand hat on. We'll still find that catalogs are an important part of the marketing mix used to educate customers about our merchandise offering.

A Great Catalog Customer

Here is my Catalog Optimizer for a "Great" catalog customer. Tell me the story the data tells you: The current strategy o...