AI in health must be the servant not the master: Accountability, evidence and decision-making autonomy
CPAI Guest Talk with professor Angela Ballantyne (University of Otago).
Abstract
In this talk I use Iain McGilchrist’s metaphor of ‘the master and his emissary’ to examine the ethical shape of human–AI collaboration in the delivery of modern healthcare. According to McGilchrist’s account of the two brain hemispheres, the emissary (explicit, decontextualized, analytic) is designed to serve the master (holistic, interconnected, meaning-oriented), yet in certain contexts risks usurping this role.
I argue that clinical AI systems – diagnostic tools, predictive models, and AI scribes – can serve as powerful emissaries, but we should be wary of AI-tools scrumptiously driving decision-making. Recent regulatory innovations try hard to ensure AI remains in this emissary role, by insisting on a ‘human in the loop’ model, clinical authority and accountability for an AI-assisted decisions. But what actually happens in practice? Particularly in busy, resource-stretched healthcare environments? AI tools are at risk of displacing clinician judgement and attenuating relational care. In this talk I will explore this ‘servant-master’ dynamic with reference to: AI literacy, automation complacency and bias, clinical de-skilling, fiduciary duty of care, and financial compensation systems for AI error. I argue that it is unfair to hold health providers morally and legally responsible for AI-promoted decisions they don’t have the time, cognitive bandwidth, or epistemic foundations to evaluate. I present next steps for next steps for conceptualising and sustaining an appropriate balance between AI and human decision-making in healthcare.
Speaker bio
Angela Ballantyne is Professor of Bioethics at the University of Otago, New Zealand. She has held roles with the World Health Organization in Geneva, Yale University, and the National University of Singapore. She has also served as President of the International Association of Bioethics and contributes to national expert groups on AI, research ethics, reproductive ethics, and immunisation policy. Her research interests include exploitation, research ethics, vulnerability, ethics of pregnancy and reproductive technologies, and secondary use research with clinical data.
Registration
Please, register to attend the talk.
Doors open at 18:30
Talk begins at 19:00