Secret Shopper / Competitive Research

Early in my career when I worked at Lands' End we focused on stuff that was within our control. We'd analyze each catalog spread like our lives depended upon it (and while our lives didn't depend on it our careers kind of did depend upon it). Our circulation director micro-managed circulation managers to the point of exasperation ... "find out from Finance if your assumption of a 59% gross margin is 59% or 59.5% and if it is 59.5% redo the entire seasonal plan by Monday morning". There was a lot of work being done, and it was all based on how "we" performed. We responded to ourselves.

Then I arrived at Eddie Bauer.

There was a team at Eddie Bauer responsible for our Secret Shopper program and Competitive Research program. We had a younger employee ... and this poor guy ... it was his job to go into Gap and J. Jill and J. Crew and take pictures of the assortment without being noticed. He'd occasionally buy merchandise so that we could analyze it and figure out how we stacked up against Gap on the exact same item. He'd occasionally be chased out of the store by a wily store manager who "knew" who he was and what he was up to.

Assuming this poor guy didn't end up in jail, he'd put together presentations showcasing how our competitors presented the same merchandise we sold ... he'd analyze price points and presentation style, setting off an in-house firestorm of criticism ... "just because they sell that item at $49.99 doesn't mean our $59.99 price point is wrong, it just means we know how to bundle more value out of the item than they bundle." Those meetings were a real treat ... as opinions crashed up against opinions. Not enough data to conclude anything, just enough data to rile up everybody. We'd be at $59.99 and 20% off (meaning we were really at $47.99), they'd be at $49.99 at full price. Who was "right"?

Our Market Research Team would combine secret shopper data with competitive research / surveys. They'd ask me to link PRIZM Cluster data to this information. We had personas ... and wait for it ... wait for it ... the best persona was a woman named "Karen" ... and they'd convert all findings into what Karen believed. You can only imagine what this was like in a meeting ... "How do you know that Karen feels this way, how could you possibly know?"

Where am I going with this?

All data / information has two components.

1. Weight / Importance.
2. Half-Life.

Making a decision based on how a person feels about a visit to your website has minimal importance and virtually no half-life.

Making a decision based on the fact that first-time buyers are always responsive in the first 0-3 months following a first order has high importance and a long half-life. In other words, you can make your company a fortune here, even if the work isn't glamorous or popular or isn't something that your "competitors" are doing.

Focus on important stuff with a long half-life. Seductive analytics based on weak information gets you nowhere.