AI Pragmatism: 8 People-Centered Questions to Help Senior Executives Overcome AI Hype
Recently I posted thoughts for senior executives in either government or the private sector facing increasing challenges of cybersecurity — and why checklists alone are insufficient for the zero trust era we’re now navigating. This week, for my 100th LinkedIn article since the series started, I’d like to dive into a topic nearly everyone is discussing in 2024 — AI in the enterprise — from the perspective of what questions senior executives should ask vendors before potentially purchasing an AI solution for an enterprise environment.
AI Has Had Multiple “Waves” of Methodologies
AI has had multiple methodologies over the last several decades since the late 1950s. Over the years I’ve had a chance to work with “expert systems” and prototypes of simulations associated with natural phenomena, fused with observations from space assets back in the mid-1990s. Later regarding counter-bioterrorism efforts, I did work involving non-obvious relationship analysis (hat-tip to Jeff Jonas and his great work there) and decision support systems associated with outbreak response .
Later in 2013, I pitched to DARPA that we could use machine learning to build a library of known antigens and associated therapies, to include antibodies, antibiotics, bacteriophages, and other treatments, in case we needed to respond to pathogens of unknown origin with speed. Then in 2016 I wrote an article positing that:
… the potential benefits of AI to our nation and world are primarily in the civilian domain. The U.S. should help pioneer and show the world how AI can be used to make people more free, prosperous, and secure in keeping with our Constitution.
More recently, I have briefed different organizations regarding the need for effective deterrence solutions to address bad (human) actors that actively misuse and abuse GenAI to erode civil norms, laws, and national defense. Also, earlier this week I participated in a lively National Academy of Public Administration (NAPA) discussion on “AI Governance” at both State and the Federal level of the United States.
So, I come at the AI discussion nowadays a bit amazed at the recent hype and doomerism associated with AI — though I also recognize the same hype and fear cycles happened with the “disruptive technology” called radio in the late 1920s and early 1930s — and I remain hopeful we’ll soon get to more pragmatic AI activities.
AI in the Enterprise Starts with Identifying Clear Business Needs
There’s a very real risk that organizations seeking to adopt AI will begin with the technology first and not the business needs first.
It’s not that an organization — be it in government or the private sector — needs to have complete clarity on its business needs before beginning an AI journey (and it is a journey, including the necessary actions to help encourage an “Always Be Learning Environment” in the enterprise, aka A.B.L.E.). However, if the metaphorical AI technology cart is placed before the more important “business need” horse, any enterprise runs the risk of a technology purchase operating disconnected from the larger “why this matters” context needed for any sociotechnological endeavor to succeed.
When considering an AI purchase for an enterprise, I’m going to pull key Bottom Line Up Front (BLUF) take-aways from three papers and strive to synthesize a consensus across all of them. The first one is the recent “AI Buying Guide for Government Agencies: What to Know Before You Buy” from Two Six Technologies, which I find applicable to private sector firms as well. The second document includes written take-aways from a video webinar with colleague Ray Wang and I hosted by MIT Sloan on “5 Steps to ‘People-Centered’ Artificial Intelligence” and the third document was published by another great friend and colleague, Vala Afshar, in ZDNet that interviewed two brilliant individuals Teresa Carlson and Rhonda Vetere, in addition to myself on the need for “When Deploying GenAI at Scale, People Must come First. Here’s How”.
The Two Six “AI Buying Guide” was written after the recent Executive Order (EO) 14110 on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence — and rightfully calls out the need to discern real value from marketing hype in a time when so many tools are branded as ‘AI-enabled,’ or ‘powered by machine learning.’” The document provides twelve key questions for senior executives to ask vendors about AI — the first six of which I find particularly beneficial in any organizational context. Consider the first three:
1. Can you share a few examples of practical use cases for your technology?
2. What types of expertise do we need (data science, programming, prompt engineering, etc.) to enable AI and adopt your solution?
3. What data are required for your solution, and who is responsible for providing it?
By focusing on the core triangle of “business need” use cases, talent required to sustain a solution, and the data sets required for the solution (and how the data will be sourced) — this “AI Buying Guide” helps separate the proverbial wheat from the chaff in terms of extant AI solutions being pitched by vendors to enterprises. In 2020, as part of the MIT Sloan webinar, Ray Wang and I similarly suggested “Classify what you’re trying to accomplish with AI” to include:
- Automate tasks with machines so humans can focus on strategic initiatives.
- Augment — applying intelligence and algorithms to build on people’s skill sets.
- Discover — find patterns that wouldn’t be detected otherwise.
- Aid in risk mitigation and compliance.
We also advocated “Establish data advocates,” and “Embrace three guiding principles” — namely Transparency, Explainability, and Reversibility with regards to the systems adopted. Ray and I also noted in 2020 that often deep learning algorithms themselves may be opaque om terms of explainability, however there are methods that human governance and oversight can do to augment this to achieve holistic Transparency, Explainability, and Reversibility as part of adopting an AI solution for an enterprise. I’ve written multiple times on LinkedIn on why trust — defined as the willingness to be vulnerable to the actions of an actor one cannot directly control (be the actor an individual, group, company, government, society, or machine) must be addressed in any sociotechnical endeavor.
AI Deployments Require Security, Model Training, and Workforce Empathy
The second set of three questions included in the “AI Buying Guide” encourage senior executives to evaluate how well a prospective vendor understands the need for data security, model training and evaluation, as well as an appropriate fit between the chosen AI methodology and tasks to be addressed; specifically:
4. Where is the solution/data hosted, and who is responsible for security?
5. How do you train and test your models for accuracy?
6. Why is AI the best choice for the tasks at hand?
Relatedly, the ZDNet article advocating for “AI deployment empathy” on the part of senior executives, colleague Vala Afshar noted: “98% of IT leaders report facing challenges regarding digital transformation. Key drivers are the persistence of data silos at 81% and the fragility of tightly coupled and highly dependent systems at 72%.”
In that same article, two tech luminaries Teresa Carlson and Rhonda Vetere emphasized that AI efforts need to achieve, especially in the public sector agencies “full interoperability with legacy systems” and to consider workforce perspectives “about where AI automation makes sense while clearly communicating reskilling plans” for extant employees of an organizations too.
My own observations included in the ZDNet “AI deployment empathy” article match those of both Two Six as well as Teresa and Rhonda, specifically that while moving fast, leaders cannot forgo security, customer value, and business continuity. Senior executives must balance thoughtfulness, empathy, and care for people with the urgency to innovate for shared prosperity.
Circling back to the topic of security — with AI systems connected to the internet, and even AI systems air gapped yet being fed data from external sources — it’s important to practice what Ray Wang I called in our MIT Sloan webinar as “Mindful Monitoring”. Two Six says the same with their last question cautioning folks about the reality that data poisoning, astroturfing, and inauthentic bot activities can be used to confuse algorithms and result in outcomes not intended by an organization (* = note that for Two Six this is their number 12, however I’ve listed here as the seventh key question I recommend asking). Specifically, senior executives need to ask vendors:
7*. How is your AI/ML resilient against technology changes or outside efforts to mislead your solution?
Lastly, I’d suggest a final question too — one that isn’t on most vendors radars yet, however discerning AI solution interoperability with other solutions will become key for large enterprises in both the public or private sector. Already for healthcare and AI efforts, there’s a real risk that an AI system will help a clinician generate patient notes *as outputs* from bullets composed by a human and then later use that output as *inputs*, risking the introduction of model instabilities to an AI system. Discerning consistent, standardized approaches to AI interoperability will be essential both in healthcare and other domains where AI-assisted outputs may be ingested later by similar or different AI systems as inputs:
8. How does your AI system support interoperability with other AI systems different from the ones you provide?
As a closing to this post, I’ll include links to all three articles at the bottom, as well as an invitation for interested folks to join a live upcoming episode of DisrupTV this Friday, 12 July at 1400/2pm ET when I’ll be joining Ray, Vala, Teresa, and special guest Miriam Vogel of EqualAI to discuss “AI Trends in Government, Venture Capital, and Global Activities”.
In the meanwhile, what recommended questions do *you* recommend for senior executives considering AI solutions for an enterprise environment? And what questions do you think senior executives should ask of vendors when evaluating AI systems?
Onwards and upwards together!
- Resource #1: AI Buying Guide for Government Agencies: What to Know Before You Buy
- Resource #2: 5 Steps to ‘People-Centered’ Artificial Intelligence
- Resource #3: When Deploying GenAI at Scale, People Must come First. Here’s How
