Yesterday, I told you a story about my time at Eddie Bauer ... we made a major creative change, the change sunk sales by 10% or 15%, +/-, and that didn't go over well with the folks who have their thumb on the pulse of the business.
In that example, after a period of time, those who made the creative changes found themselves exploring new opportunities.
But what if folks had the patience to wait longer?
A few years ago, a person working at a major retailer told me a story about a change in creative. The website was literally re-built, from scratch ... new platform, new creative presentation, new imagery, new user-generated content. New everything.
In month one, conversion rates were -15%.
In month two, conversion rates were down -10%.
In month three, conversion rates were down -5%.
In month four, business reverted back to normal.
Imagine if you A/B tested this thing for a month ... came up with a statistically significant outcome (new = worse #measure #datadriven #gutinstinctisforlosers), and told the company to not make the change?
In 1998, I would have been a punitive moron ... I would have not appreciated changes that did not immediately produce positive results. In 2015, I hope I'd be more nuanced. I'd probably use my comp segment framework to demonstrate that, at this time, the change is not working ... and then, I'd give Management the two data points described in the past two blog posts as examples of what could happen.
Truth is, we don't know, at month one, whether the -15% will become 0 or will stay at -15%. Creative always requires a certain amount of faith, and a certain amount of prudence is needed before pulling the plug.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.