Unpopular opinion: executives don't need "data literacy" training
Every few months, I get this question on my inbox: Would you be willing to design and teach a quick data literacy course for the executives at my company?
Typically the request comes from a senior manager from a traditional company hoping to get more buy-in for their goal to improve core operations using machine learning. And the requester is not alone in thinking that executives should become more fluent in data and analytics concepts. For example, in the book Keeping Up with the Quants: Your Guide to Understanding and Using Analytics, Thomas Davenport writes:
"Some of the concepts that any executive needs to understand include:
Measures of central tendency (mean, median, mode)
Probability and distributions
Sampling
The basics of correlation and regression analysis
The rudiments of experimental design
The interpretation of visual analytics
The methods for acquiring this knowledge can be the same as for more junior personnel, except that senior executives may have the resources to bring in professors or consultants for sessions with groups of executives, or even private one-on-one tutoring."
There are various schools of thought of what a “data literacy for executives” program should look like. Some offerings are more focused on traditional analytics, covering dashboards, KPIs, and data visualization; others concentrate on developing “AI fluency”, which is the primary goal of the people asking for my help.
Let’s review the two ideas separately: the need for analytical skills at the business executive level, and the efficacy of training to fulfill this need.
Every executive needs analytical skills.
I don’t think anyone will disagree that competencies like statistical literacy or the ability to contextualize data presented in visual formats can be valuable assets for any business decision-maker.
Still, I have worked with highly successful executives who couldn’t correctly read a bar chart to save their lives. Or who had trouble understanding why repeatedly running different statistical tests on the same dataset and only reporting the most interesting results isn’t a statistically-valid practice.
I’ve also worked with executives with the same knowledge gaps who made terrible decisions that ended up killing their startups or destroying value for their company.
That difference in outcomes can be explained by a set of behaviors that characterize the best decision-makers:
Surround oneself with advisers that 1) have the very best analytical skills; 2) can explain things in plan language; and 3) can be trusted not to spin the numbers.
Seek feedback (not consensus) from the people who are the most knowledgeable about the topic at hand—usually employees at a lower level in the organization doing the hands-on work.
Ask questions about the reasoning or argument that connects the data to an insight, or a business problem to an AI/ML strategy.
Push back when they don't understand why a recommendation was made or a conclusion was reached, if necessary asking for translations of highly technical terms into "English-speak" explanations.
In short, what someone needs in order to be a leader that creates value from data and analytics is the ability to think critically, apply first principles to problem-solving, and recognize the value of consulting with those who can contribute in a meaningful way to their decision-making process. Understanding the concept of p-hacking, or the difference between algorithms like Random Forest and CNN, is entirely optional.
Small group training sessions can be an effective method to make executives more “analytically-minded”.
The many providers of “data literacy” courses for executives will disagree with my stance on this, but let’s think about it for a moment. We’re talking about busy executives with severe demands for their time and attention. How likely is it that even the smartest of them will successfully tackle the cognitive work that understanding complex analytics concepts requires? Even under the best circumstances of extremely intelligent people being offered content sufficiently fresh, provocative, and relevant to their business, I’d estimate this probability as very low.
The one-on-one tutoring suggested in Davenport’s book is a slightly better approach, allowing for a more custom delivery of content that focuses on an individual’s specific needs. But it risks suffering from the same deficiency that group training does: attempting to squeeze too much learning into a few sessions in response to time and budgets constraints.
My verdict is in
In my experience working alongside senior management and C-level executives with an anemic experience with analytics, there’s little to be gained from sending them to a “data literacy” program in hopes that they’ll come back more analytics-minded. It’s much more effective to surround those leaders with proficient analytics practitioners who are deeply curious about the business and can customize their analytics presentations to their audience’s needs.
And for an organization considering incorporating AI/ML in some of its processes or offerings, more important than “executive AI fluency” is to make sure business leaders understand and apply the elements of great decisions. That’s what will help them make wise decisions regarding when to use machine learning to automate operational decisions and how mitigate risks and optimize ROI. And that’s also what will help them ensure the organization has all the elements in place to make analytics part of the fabric of their daily operations, embracing not only the right data and technology, but also the right processes and ongoing acquisition of analytical skills across all organizational levels.
And if you’re a business leader looking to upgrade your analytical skills or become “AI fluent”, some good starting points are the books Behind Every Good Decision: How Anyone Can Use Business Analytics to Turn Data into Profitable Insight by Piyanka Jain and Puneet Sharmaand, and Digital Decisioning by James Taylor.