“It Is a Moving Process”: Understanding the Evolution of Explainability Needs of Clinicians in Pulmonary Medicine
Published in CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems, 2024
Recommended citation: @inproceedings{Corti2024XAIIPF, author = {Corti, Lorenzo and Oltmans, Rembrandt and Jung, Jiwon and Balayn, Agathe and Wijsenbeek, Marlies and Yang, Jie}, title = {``It Is a Moving Process": Understanding the Evolution of Explainability Needs of Clinicians in Pulmonary Medicine}, year = {2024}, isbn = {9798400703300}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3613904.3642551}, doi = {10.1145/3613904.3642551}, abstract = {Clinicians increasingly pay attention to Artificial Intelligence (AI) to improve the quality and timeliness of their services. There are converging opinions on the need for Explainable AI (XAI) in healthcare. However, prior work considers explanations as stationary entities with no account for the temporal dynamics of patient care. In this work, we involve 16 Idiopathic Pulmonary Fibrosis (IPF) clinicians from a European university medical centre and investigate their evolving uses and purposes for explainability throughout patient care. By applying a patient journey map for IPF, we elucidate clinicians’ informational needs, how human agency and patient-specific conditions can influence the interaction with XAI systems, and the content, delivery, and relevance of explanations over time. We discuss implications for integrating XAI in clinical contexts and more broadly how explainability is defined and evaluated. Furthermore, we reflect on the role of medical education in addressing epistemic challenges related to AI literacy.}, booktitle = {Proceedings of the CHI Conference on Human Factors in Computing Systems}, articleno = {441}, numpages = {21}, keywords = {Explainable AI, Healthcare, User Needs}, location = {