News and views
Estimated read time: 4 mins
The AI safer practice framework is made up of two parts: INFORMED and RECORDS.
INFORMED guides ethical decision-making using AI, while RECORDS documents AI-assisted decisions for accountability and clinical rationale. The framework has been structured around these acronyms to ensure it is practical and memorable.
Advances in medicine have been driven by developments in materials science and technology.
But artificial intelligence (AI) represents something different. It is an evolving system capable of interpreting data, analysing images, predicting outcomes, and sometimes recommending interventions. The future we imagine is already pressing at our door, such is the pace of development.
At the centre of today’s debate on AI in healthcare is the question of oversight. Most current frameworks, including our INFORMED framework, emphasise human-in-the-loop (HITL) systems – AI outputs that are supervised, validated, and ultimately signed off by a human clinician. The logic is straightforward – humans bring professional judgement, context, and accountability. In this way, the dentist is seen as the strong link in the chain, ensuring that patient safety is not compromised by technology and algorithmic limitations.1
This assumption deserves closer examination. Why do we believe that humans always get it right? The medicolegal case experiences tell us otherwise. Clinicians are fallible. Diagnostic error remains one of the leading causes of harm in healthcare worldwide. Cognitive biases such as confirmation bias, anchoring, and availability heuristics distort decision-making – even in experienced hands. Stress, fatigue, workload, and commercial pressures also take their toll. The truth is that the ‘human factor’ is already a weak spot in clinical safety.2
By positioning HITL as the ultimate safeguard against AI error, we risk overestimating human reliability while underestimating human vulnerability. Take radiographic diagnosis for example, AI systems are already showing accuracy comparable to, and sometimes exceeding, that of healthcare professionals.3
What exactly is the healthcare professional’s role in overseeing these technologies? Let’s consider this under what I call the VIE model – see Figure 1 – which offers a way of framing this responsibility.
It is a continuum of development in oversight: beginning with verification as the foundation, expanding through interpretation to add clinical meaning, and extending into enablement as AI becomes more autonomous. Each stage builds on the previous one, reflecting the profession’s evolving role in safeguarding safety, trust, and ethical practice as technology advances.
Figure 1: The VIE model. A shift from verification in the present to enablement in the future.
As AI continues to improve, the difference between human and machine performance may widen. The clinician may increasingly become the weak link, slower (although that may be an asset), more inconsistent, and more error-prone than the technology. At that inflection point, the original logic of HITL starts to reverse. Instead of being the strong link that corrects machine error, the human risks becoming the weak link that introduces error into an otherwise reliable system. It is an unsettling paradox.
I must stress that I am not suggesting the trust and reliance of human oversight is misplaced. Quite the opposite. At present, AI systems remain fragile, they lack transparency and are vulnerable to bias in their training data. AI cannot fully understand the human dimensions of care – patient values, preferences, and context. The clinician is still essential, not just for validating outputs but for discussions around uncertainty of outcomes and gaining consent for example. These responsibilities cannot be delegated to algorithms.
We must also look ahead. Large language models (LLMs) and other generative AI systems are advancing rapidly. Their ability to synthesise information, adapt to context, and mimic reasoning suggests that autonomous AI in healthcare may be on the horizon.
When that day arrives, the key question will not be whether humans should stay in the loop, but whether the loop itself should be redesigned.
We cannot cling to the belief that human oversight will always be the gold standard of safety. Instead, we must be realistic about both the strengths and limitations of human judgment, and we must develop governance frameworks that can evolve as the balance of responsibility between humans and machines changes.4
We must approach it neither with blind optimism nor paralysing fear, but with reason and clarity. LLMs may also lay the groundwork for a different future – one in which autonomy in AI is an operational reality.
As I finish editing this article, I take a short break and check my emails. There is an email from an AI developer with whom I have spoken with before. The subject of the email reads ‘autonomous AI’ and attached to the email is a non-disclosure agreement.
I am reminded of Einstein’s words: I never think of the future. It comes soon enough. It just did.
References
Shortliffe, E. H., & Sepúlveda, M. J. (2018). Clinical decision support in the era of artificial intelligence. JAMA, 320(21), 2199–2200.
Topol, E. (2019). Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again. Basic Books.
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022). AI in health and medicine. Nature Medicine, 28(1), 31–38.
European Commission. (2024). Artificial Intelligence Act. Brussels.
