It might be the most asked question I get, and for good reason ...
Question: "Kevin, what the heck are you talking about when you say that you shouldn't test something that has a short half-life?"
First, let's define half-life ... go to Wikipedia for your answer!
Now let's view this from a testing standpoint. Maybe you want to test a big green "sign up now" button instead of a small blue "click here for more information" button. You run a test, and you learn that the big green button improves conversion rate by 34%.
Do you assume that the big green button will outperform the control by 34% forever?
If you are measuring "half-life", you are measuring the amount of time that it takes for the 34% lift to become a 17% ... or worse, to become a 0% lift.
So often, we test concepts that either have minimal half-life, or are only valid for the time/audience when they were tested. In other words, the concepts we are testing are fleeting in nature, they have no staying power.
There are concepts that have significant staying power. In apparel merchandising, you know that women buy merchandise for men, so you can improve conversion (online) or reduce expense (offline) by exclusively advertising womens merchandise. This has a half-life just shy of infinity, you'll be more profitable for the duration of your career by testing strategies that capitalize on this well-known fact. We don't read about these strategies and test findings, because the strategies are so profitable that they yield a significant competitive advantage to the folks who possess the knowledge.
Many concepts have minimal staying power, and, by definition, minimal half-life. We read about these strategies and test findings all of the time, because these strategies have a short half-life, rendering their competitive advantage only to the audience that saw the message during the timeframe when the test happened.
This is the phenomenon I refer to when I mention "half-life" in my writing. The best way to determine if you have a half-life issue is to re-test your strategy three months, six months, or twelve months later, observing if you still have a meaningful lift over your control group.
Maybe. If you are Starbucks, sure, your darn addictive product is more than enough to foster loyalty. In fact, when your customer purch...
Look at the first four rows of our life table (values of 0/1/2/3). These are the first 12-15 weeks after a customer buys for the firs...
So Amazon created a major shopping event out of nothing, and now they're killing it in July (a month when nobody can sell anything ot...
This is the fourteenth summer writing this blog ... let that fact sink in for a moment. As I've done in past years, expect a cadenc...