Read through each step, and ask yourself if your brand follows these steps.
Step 1: Always Execute A/B Tests. It's a good idea to make sure that some of your e-mail marketing list or catalog marketing list or paid search keyword campaign does not receive the promotion, and measure the performance of customers who receive the promotion vs. those who do not receive the promotion.
Step 2: Measure A/B Tests Long-Term. I cannot stress this enough. If you execute a free shipping test in November, re-visit the results of the test next May or June. Seriously. Measure the short-term impact and long-term impact. If you see a 20% lift in weeks 1-3 (the test period), and a -20% lift in weeks 4-26, you know that the promotion did not help your brand --- at all! Few folks actually execute this level of testing discipline. Those who do know secrets that the rest of us fail to understand.
Step 3: Quantify Incremental Orders And Flunked Orders. Incremental orders are those that were generated because of the promotion. Flunked orders are those that would have happened anyway, but now you gave away profit that lowers your bonus amount. If the test/control groups suggest that the promotion gave you a 20% increase in sales, while almost all of the orders happened with the key-code, then you know that 80% of the orders would have happened anyway. Flunked orders are bad --- they are orders where the customer believed in your brand, and would have spent hard-earned dollars that generate profit. We choose to flush that profit away. See Step 2 to understand if flunked orders generate long-term profit.
Step 4: Quantify "Staying Power". At Eddie Bauer, we measured the incremental value of the promotion by day. If the promotion lasted three weeks, we measured the lift by day. Those who execute this style of analysis know secrets about the staying power of tests that the rest of us simply make guesses about. Staying power is important, because if a promotion loses impact after a few days, you're required to constantly pull promotions off the board, then put them back up a few days/weeks later. And that isn't healthy for a brand --- it is like constantly hitting the gas pedal and then hitting the brakes while driving a car on the freeway. It means you are shifting power away from merchandise and to promotions. That's not a good thing --- you're in business to sell merchandise, not to sell promotions and gimmicks.
Step 5: Understand Who Utilizes Promotions. We don't do nearly enough of this, do we? Good analytics teams profile the customers who take advantage of promotions. Are they the best customers? Full price customers? Sale customers? Promotional customers? New customers? Lapsed customers? After profiling the audience, measure the long-term value of the customers utilizing promotions, comparing them to the customers who pay full price and pay for your bonus check.
Step 6: Understand Channel Dynamics. There's nothing wrong with executing promotions in e-mail campaigns and not in other advertising channels. But it is important to understand what this does to your business. Do the promotions cause e-mail subscribers to not buy from full-priced catalog marketing? If so, adjust the contact strategy accordingly, and realize that your e-mail marketing strategy lowers response in your catalog marketing channel (or vice versa). And seriously consider why your e-mail subscribers are more deserving of discounts and promotions than all other customers.
Okay. So I fully believe what you've outlined here are the smart and right things to do. Also, I would like to do this for all promotions. However, as a one-person analytics team for a brand that is not following these steps, I'm feeling a little overwhelmed. I won the "A/B test the catalog" argument, so I'm getting ready to make time for those upcoming analyses. But with so many things to look at, in order to do the kind of whole-picture measurements that you describe, where do I start? Honestly, I'm not sure I could do all of this and keep up with all of my normal duties! It's sad because not doing things the right way does the company (and ultimately myself) a disservice.
ReplyDeleteFirst of all, kudos to you! You are in the majority.
ReplyDeleteWhen I work with folks in your situation, I recommend doing a test plan. If you only have resources to do three tests and one analysis a year, pick the issues that are most important, and spread each one out over a three month period of time.
This gives you the opportunity to test the things that matter most. And if you only have resources to do one analysis, do that one analysis. Promote the fact that you're going to do these four things, and then use the results of these four things to shape the four things you are going to do in 2010, etc.
Maybe you have the resources to do that, or maybe you can only do 1-2 things a year. But by doing just those 1-2 things, you are ahead of many other companies. And you're learning, too, which is good!
I'm proud of you!!
As always, excellent advice. Thank you, Kevin!
ReplyDelete