Analytics Failure

Last night in the NFC Championship Game the (checks notes) Detroit Lions led in the 3rd quarter by 14 points. They had a 4th down in field goal range. The didn't kick a field goal that could have put them up by 17 points. Instead, they went for it on fourth down (in the spirit of the fearlessness they exhibited all season) ... and they failed.

From there, the script writers put Detroit fans through a predictable blender of negative emotions culminating in a 34-31 loss to the 49ers.

A debate raged thereafter ... "analytics" suggested Detroit made the right decision. If one goes back an analyzes a wide range of games, the odds of winning increase more by going for it than by kicking a field goal.

Of course, analytics people fail the general public all the time. In this case, "analytics" should be reserved for young teams playing on the road in a Championship Game. It doesn't matter what Carolina did up 14 against the Cardinals in a game five years ago in a comparable situation in early October. It matters what a young team playing on the road in a Championship Game does in that situation.

One of my favorite analytics failures happened more than twenty years ago when the entire catalog industry was being forced into a digital world. When you acquired an online buyer, the online buyer had lower "future value" and therefore "analytics" told leadership to instead focus on a shrinking base of traditional catalog buyers. There were two problems with this thesis. First, digital customers had lower future value (profit) because they were being mailed wasteful catalogs they had no interest in whatsoever. Second, there was no data to determine the outcome of what would happen if a catalog brand continued to invest in a shrinking audience (which, by the way, was a terrible idea ... leading to a 65+ year old customer base in 2024 that will never migrate to digital tactics while only preferring merchandise that caters to 65+ year old customers).

In other words, "analytics" were wrong because the conditions being measured by analytics did not replicate the specific situation being analyzed.

Same thing with the misguided "omnichannel" approach to business that harmed retail brands. The analytics showed that buying from multiple channels was good. The analytics were wrong, of course, because the analytics were not built off of a specific scenario where existing channels were dying in favor of emerging channels ... analytics assumed a constant scenario. This mistake led to a "retail humbling" that we're still dealing with today.

Just because somebody is good with "analytics" doesn't mean that somebody understands how to apply "analytics" to a situation.