AI Leadership in Healthcare Must Be Governed and Physicians Play a Key Role

Artificial intelligence is no longer a future concept in healthcare. It is already shaping how we document care, triage patients, allocate resources, predict risk, manage revenue cycles, and increasingly, how clinical decisions are supported … And most of this is done beneath surface.  What remains unsettled is not whether AI will be used … but how, when, by whom, and under what standards?

That question places physicians at the center of a defining leadership moment.

Healthcare has always relied on physicians to steward and apply powerful tools responsibly. From new surgical techniques to pharmaceuticals to electronic health records, clinicians have served as the moral, clinical, and professional compass when innovation outpaces policy. AI is no different … except for its scale, speed, and reach across the entire healthcare enterprise.  This is bigger more than a novelty … AI will be an important as cash and may need to be managed as such.

If physicians do not lead AI governance, others will. And those others may not fully understand the clinical nuance, ethical stakes, liabilities, or unintended consequences of algorithmic decision-making in patient care.

AI Is Not Just a Technology Issue … It Is a Top-Level Leadership Issue!

Many healthcare organizations still approach AI as an IT, analytics, or vendor problem. In reality, AI is a system-level resource that affects culture, strategy, patient safety, workforce dynamics, equity, risk, and trust.

AI already influences:

• Which patients are flagged as high-risk
• How documentation is created and interpreted
• How staffing, throughput, and utilization are optimized
• How clinical variation is managed, or amplified
• How patients experience transparency and consent
• Revenue optimization
• Claims and coding
• Unofficial advising at every level of the organization

These are not technical decisions alone. They are leadership decisions that influence how we support the mission, and physician leaders must be present where those decisions are made.  The real concern is that these are just the things we know about AI in our healthcare systems.  In my surveys, I am finding that almost 100% of early and mid-level clinicians/administrators use AI, but less than 2% have formal training and only 20% share their AI use with their leadership.  If I were a CEO again, that level of risk would keep me up at night.

To assure we are leading this monumental transformational change the right way, we need to prepare our physicians to ask the right questions of leadership.  A few foundational questions physicians should be asking early and often of their C-suite is:

What governance structure is in place specifically for AI, and how is it embedded into daily operations?

Physicians should be asking who owns AI oversight, how responsibilities are distributed across clinical, operational, compliance, and technical teams, and how governance decisions are enforced in real workflows … not just on paper.  Who is responsible for our AI systems?  We cannot let just the CIO or CFO oversee a process that has potential to impact ALL of our patients and our system’s / vendor’s processes. AI exceeds their roles. AI’s “footprint” is so large, it must be elevated and led by the CEO and reported as a special item to the Board … similar to the budget.  It must also be part of a revised strategy for the organization.

What problem(s) are we using AI to solve, and how will we know if it is truly helping?

Without clarity of purpose, AI tools risk solving the wrong problem, optimizing for convenience rather than outcomes, or introducing complexity without value. Physicians are uniquely positioned to demand alignment between AI use and meaningful clinical or operational improvement.  We must insist on clear success metrics and accountability when outcomes fall short. Shared learning across the system is essential with AI.

How are we training our staff on the value, risk and ethical use of AI? AI Literacy Is now a core physician competency … and it should be for everyone.

Physician leaders do not need to become data scientists. But they do need to understand enough to ask the right questions.

That includes understanding:

• Common types of AI such as machine learning, natural language processing, and generative models
• Where AI is being used in both clinical and non-clinical workflows
• How data quality, bias, and training sets affect outputs
• Where limitations and uncertainty exist
• What do we do when it fails? Who is liable?
• How do we make AI transparent?

This literacy enables a second essential question at the executive level:

Where is human judgment required?  and where is it being replaced?

AI should augment clinical and operational decision-making, not quietly displace professional judgment. Without explicit guardrails, automation bias can creep in, and clinicians may defer to algorithmic recommendations even when experience or context suggests otherwise. Physicians must help define where AI advises, where it assists, and where it must never decide alone.  That must be standardized across the system.

Governance must be operational, not theoretical

AI governance cannot live in a policy binder or a quarterly committee meeting. It must be embedded into daily operations, how tools are selected, deployed, monitored, retained, and, when necessary, retired.

National standards are emerging to guide this work in the next month that will be provided an AI governance framework developed and approved through ANSI processes. This framework will emphasize multiple domains of oversight, including accountability, transparency, data integrity, human oversight, safety, ethics, training, and continuous monitoring. It is a first-step in creating “guardrails” to mitigate the risks and assure success for AI in healthcare.

Because these risks evolve, physicians must ask:

How are bias, equity, and patient safety being monitored over time, not just at launch?

Initial validation is not enough. Healthcare environments change. Data changes. Patient populations change. Continuous monitoring is essential to ensure AI tools remain safe, equitable, and clinically appropriate across settings and over time. Who is assuring our data is “clean”?  We have neglected to invest the resources to assure timely, accurate data historically.  It may now be essential.

Culture, trust, and the physician voice

AI adoption is also a cultural transformation. How AI is introduced, explained, and governed sends a powerful message to clinicians and patients about transparency, respect, and professional autonomy.

Physician leaders should be asking one final, foundational question:

How are we protecting trust with our clinicians and with our patients?

This includes transparency about when AI is used, how data is handled, how decisions are explained, and how concerns are surfaced and addressed. Trust, once lost, is extraordinarily difficult to rebuild, especially in healthcare.

This is important because AI’s promise comes with real potential and risk

AI has extraordinary potential:

• Reducing administrative burden
• Supporting earlier diagnosis and intervention
• Improving coordination and access
• Enhancing efficiency across the system
• Optimizing system resources

But it also carries significant risk:

• Embedded bias that worsens disparities
• Model drift that degrades accuracy over time
• Opaque “black box” decision-making
• Workforce disruption and deskilling
• Erosion of patient trust
• Workforce degradation
• Harm

From passive users to active stewards

The future of AI in healthcare should not be something that happens to physicians. It must be something they actively steward.

That stewardship includes:

• Key role in AI governance oversight
• Participating in AI procurement and vendor evaluation
• Guiding cross-functional collaboration
• Understanding legal and compliance implications
• Championing ethical, transparent use
• Developing personal leadership action plans to advance AI literacy and governance

This moment does not call for resistance to technology. It calls for leadership rooted in medicine’s enduring values, judgment, accountability, and care for those we serve and those that serve them.

AI will continue to advance. The real question is whether physicians will help shape its role, or be asked to adapt after decisions are already made. Healthcare needs physician leaders who can confidently sit in the C-suite and boardroom, ask the right questions, and ensure that AI augments our team and serves medicine, not the other way around.

And that leadership must start now … AI is already on your team, you just don’t know it.