Sharing Information With Executives

Yesterday, I mentioned to folks on Twitter that I used three different analytics tools to measure the same thing. I was trying to quantify how many people viewed a page.

• Tool #1 Counted 15 "hits".
• Tool #2 Counted 10 "hits".
• Tool #3 Counted 9 "hits".

I offered the comment "... how do we trust information when three tools give us different answers?" I also mentioned that CEOs don't like different answers from different sources, they like for information to tie out.

It was the latter comment that frustrated a few analysts.

Last week, we talked about the gulf between analysts and executives, in fact, this was one of the most popular posts written this year. But as Mr. Peterson would say, people like to grumble about things, but it is much harder to do something about it.

So let's start doing something about it!

Let's review a series of topics about analytics that frustrate business leaders, and discuss what we can do to bridge the gap a bit.

Executive Pet Peeve #1 = Numbers That Don't Tie Out: Always remember that business leaders get reporting from sources that are reasonably "iron-clad". The CFO says that net sales were $22,000,000 in October and profit was $1,500,000. The Chief Merchant says that she sold exactly 2,493 units of an item. All day long, the business leader is dealing with numbers that "tie-out", they match metrics found in other sources.

Then here comes you, the web analytics expert, sharing that your reporting says that net sales were $19,000,000 in October, because your web analytics software package doesn't capture all sales. Granted, your did some spectacular work, and found many interesting customer trends in your analysis. But as soon as you leave the boardroom, your CEO will look at your CFO and say "... Ralph's numbers don't tie out with our order entry system or with finance numbers, how can that be? Can we trust Ralph's analysis?"

You already know that Alexa and Quantcast yield different outcomes, and you already know that Coremetrics and Google Analytics yield different outcomes. You can explain the differences, you are comfortable with the differences. But your executive team does not have to deal with this level of uncertainty with, say, merchandise reporting (though honestly, other sources don't tie out either, they just won't share that fact with you).

Solution: Know the biases in your data, and willingly share them before people start to question the data. Maybe your web analytics solution misses 15% of sales. Match this information with your order entry system (or have a tech person do this for you), and understand where the sales that don't tie out come from. Then tell a story that matches the worldview of your executive team. For instance, if you are missing sales, it is fine to confess that fact, saying "Well, we've thoroughly audited this problem, and it turns out that our analysis is disproportionately focused on our best customers, and as you know, one of the three biggest initiatives for 2009 is to get more sales out of best customers, so the data I am presenting supports a big company initiative." Also, take all references to any triangulation you did out of the spotlight. In other words, if you are analyzing data from different sources, clearly state in your documentation/appendix that you did that, and clearly state what the biases are. But you are the expert, so sell your story to your leadership team, earn their trust by being confident that directionally, you are recommending a great strategy. And then, be right! If your conclusions are consistently right, executives will stop asking questions about data integrity!

Executive Pet Peeve #2 = Misaligned Objectives: Your executive team isn't going to tell you all of the things they are focusing on. It is possible that they are being pressured by the credit division to increase credit utilization or the credit division will have to sell the portfolio, so the executive team asks you to analyze conversion rates among proprietary credit card holders on your site. They simply don't give you any background about the request, other than to produce a report for them. And you do a great job on the analysis, showing that credit card holders are converting less and less often (maybe because they've maxed-out their credit limit). This answer may not be aligned with what the executive team needed to hear --- they gave you a task and were hoping for one outcome, you provide them with another outcome. And then, you try to complement your analysis with a recommendation to drive non-proprietary credit traffic via paid search, in an effort to optimize conversion rate. Your recommendation, while spot-on, is not what the executive wants to hear. And as a result, the executive is going to do one of three things ... the executive is going to have you go on a fishing expedition to find one metric that meets her needs (a very frustrating process, if you are an analyst), or the executive will tell you that something is wrong with your analysis, or the executive will politely thank you for your work, without telling you that you failed to meet her expectations.

Solution: Ask questions when you are assigned a project. Ask the executive if there are any land mines to look out for. Ask the executive to provide you with any background information that can help you provide a better analysis. Ask the executive what the "expected outcome" of the analysis is. Ask the executive how the executive will judge if you did a good job? Ask the executive if there is an "extra credit opportunity" ... in other words, are there follow-up analyses and projects that you could do that would even better compliment the work? After doing this a few times, you will get a feel for what your executive sponsor is "really" looking for, and you'll do a much better job of completing future projects. The responsibility for ferreting out information is on you, dear analyst, because executives are not going to ever volunteer enough information to inform your work ... in fact, in some cases, they simply cannot tell you the real reason for an analysis (i.e. they need numbers to determine if 5% or 10% of the workforce will be let go next month).

Executive Pet Peeve #3 = Loyalty. I once had an employee who sent me an e-mail, suggesting that I purchase a service from a vendor. At the end of the e-mail message, the employee quoted a vendor tagline. I later learned that the employee brought in a vendor for an informal visit, listened to the sales pitch, and then was recommending the vendor to me. I knew that the employee was loyal to the vendor, and not to me. I was an executive at Nordstrom, and without a doubt, I expected my analytics employees to be more loyal to me than to vendors that were pitching products and solutions to them. I expected my employees to be loyal to me, to my department, to my company, and most of all, loyal to the "voice of the customer", communicated via solid analytical rigor.

Solution: Your company might truly need a product or solution from a vendor. Your job, as an analyst, is to provide the business case for the product/solution. How would the product or solution or analysis help the executive? Not surprisingly, executives are looking for solutions that allow them to keep their job. I mean, take a look at many executives, they go from company to company, being hired and fired every few years. Most business leaders want to keep their job, and are under a lot of pressure to do so. So make it easy for them. Any analysis, product, or solution that makes it easier for a business leader to keep her job and to look good among executive peers is an analysis, product, or solution that is going to be used. You can be totally honest, offering all of your own unique ideas and solutions in any work you do. But first, demonstrate to the executive that you are there to help the executive, then provide facts/analytics that support the business leader, and then in an appendix, go ahead and share all of your own ideas. If the executive knows you are loyal to the executive, then you are likely to be called upon more often. This doesn't mean you are "sucking-up" to the executive. Nope, it means quite the opposite. It means you are respecting the needs of the company, still offering your point of view in an appendix. Eventually, most executives will trust you, and when that happens, look out! They won't question you as much, they will listen to your recommendations, and they will support you publicly with other business leaders. At that point, you are in a position to enact change!

Executive Pet Peeve #4 = Errors. An executive wants every number to be accurate. Any error, even if it isn't your fault, becomes your fault if you present it incorrectly as part of a project for an executive. Executives demand accurate information.

Solution: Audit every single piece of work you do. Demonstrate somewhere (an appendix) that your numbers tie out to another source. Ask somebody else to look over your work.

Executive Pet Peeve #5 = Agenda. An executive wants a loyal employee. But sometimes, loyal employees (I've done this) develop an "agenda". And if the agenda is not congruent with the executive, look out. Maybe you want to do multivariate testing on the home page, but the executive you work for is having a battle with another executive in another department, and wants to "be right". In this case, even though you are doing the right thing (by wanting to execute a multivariate test), you might appear to have an agenda (i.e. prioritizing testing over the executive). And once you appear to have an agenda, you are banished to the island of misfit toys. You'll see this happen all of the time when a new executive takes a job --- the new executive has a vision for what she wants to accomplish, and if your agenda is not aligned with the new executive, it doesn't matter how good of a job you do, you're going to be banished to the island of misfit toys.

Solution: Know your corporate culture! This sounds like bad advice, but it is important. For instance, your culture may support testing, but may not be receptive to the findings derived from tests. In that case, test like crazy, and gain as much knowledge as possible, so that you will have the facts when a business leader is ready to accept them. If you do have an agenda that is not congruent with your business leader, consider implementing your agenda on different projects with different business leaders, if your executive is supportive of that. Or, consider working for a different executive altogether (or a different company), if your agenda is different from that of leadership.

Ok, there's a few issues and solutions for you to consider. It's been my experience that business leaders are not going to volunteer a lot of information, and sometimes do not want to hear some of the things you are talking about. They are probably not going to meet you half-way, so why not try to find out what they need?

Now, it's time for you. What wasn't addressed here that should be addressed in a subsequent post? Leave your ideas in the comments section of this post, thanks.