The Role of XAI in Advice-Taking from a Clinical Decision Support System: A Comparative User Study of Feature Contribution-Based and Example-Based Explanations
Source abstract
Abstract
Explainable artificial intelligence (XAI) has shown benefits in clinical decision support systems (CDSSs); however, it is still unclear to CDSS developers how to select an XAI method to optimize the advice-taking of healthcare practitioners. We performed a user study on healthcare practitioners based on a machine learning-based CDSS for the prediction of gestational diabetes mellitus to explore and compare two XAI methods: explanation by feature contribution and explanation by example. Participants were asked to make estimates for both correctly and incorrectly predicted cases to determine if there were any over-reliance or self-reliance issues. We examined the weight of advice and healthcare practitioners’ preferences. Our results based on statistical tests showed no significant difference between the two XAI methods regarding the advice-taking. The CDSS explained by either method had a substantial impact on the decision-making of healthcare practitioners; however, both methods may lead to over-reliance issues. We identified the inclination towards CDSS use as a key factor in the advice-taking from an explainable CDSS among obstetricians. Additionally, we found that different types of healthcare practitioners had differing preferences for explanations; therefore, we suggest that CDSS developers should select XAI methods according to their target users.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Explainable Artificial Intelligence (XAI) · Artificial Intelligence in Healthcare and Education · Machine Learning in Healthcare
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
UCD School of Computer Science, University College Dublin, D04 V1W8 Dublin, Ireland
UCD Perinatal Research Centre, School of Medicine, University College Dublin, National Maternity Hospital, D02 NX40 Dublin, Ireland
Funding
University College Dublin and National Maternity Hospital Medical Fund
References (35)
Show source-provided references
1. An overview of clinical decision support systems: Benefits, risks, and strategies for success10.1038/s41746-020-0221-y ↗
2. AI in health and medicine10.1038/s41591-021-01614-0 ↗
3. A machine learning–based clinical decision support system to identify prescriptions with a high risk of medication error10.1093/jamia/ocaa154 ↗
4. Ragab, M., Albukhari, A., Alyami, J., and Mansour, R.F. (2022). Ensemble deep-learning-enabled clinical decision support system for breast cancer diagnosis and classification on ultrasound images. Biology, 11.10.3390/biology11030439 ↗
5. A clinical decision support system for predicting invasive breast cancer recurrence: Preliminary results10.3389/fonc.2021.576007 ↗
6. Development of machine learning-based clinical decision support system for hepatocellular carcinoma10.1038/s41598-020-71796-z ↗
7. Antoniadi, A.M., Galvin, M., Heverin, M., Hardiman, O., and Mooney, C. (2021, January 22–26). Development of an explainable clinical decision support system for the prediction of patient quality of life in amyotrophic lateral sclerosis. Proceedings of the 36th Annual ACM Symposium on Applied Computing, Virtual Event, Republic of Korea.10.1145/3412841.3441940 ↗
8. Deep-spindle: An automated sleep spindle detection system for analysis of infant sleep spindles10.1016/j.compbiomed.2022.106096 ↗
9. Machine learning based decision support systems (DSS) for heart disease diagnosis: A review10.1007/s10462-017-9552-8 ↗
10. A systematic literature review of AI-based digital decision support systems for post-traumatic stress disorder10.3389/fpsyt.2022.923613 ↗
11. Mazo, C., Aura, C., Rahman, A., Gallagher, W.M., and Mooney, C. (2022). Application of Artificial Intelligence Techniques to Predict Risk of Recurrence of Breast Cancer: A Systematic Review. J. Pers. Med., 12.10.3390/jpm12091496 ↗
12. Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., and Elhadad, N. (2015, January 10–13). Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia.10.1145/2783258.2788613 ↗
13. Antoniadi, A.M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B.A., and Mooney, C. (2021). Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review. Appl. Sci., 11.10.3390/app11115088 ↗
14. World Health Organization (2022, October 04). Ethics and Governance of Artificial Intelligence for Health: WHO Guidance. Available online: https://www.who.int/publications/i/item/9789240029200.
15. Panigutti, C., Beretta, A., Giannotti, F., and Pedreschi, D. (May, January 29). Understanding the Impact of Explanations on Advice-Taking: A User Study for AI-Based Clinical Decision Support Systems. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA.10.1145/3491102.3502104 ↗
16. Antoniadi, A.M., Galvin, M., Heverin, M., Wei, L., Hardiman, O., and Mooney, C. (2022). A Clinical Decision Support System for the Prediction of Quality of Life in ALS. J. Pers. Med., 12.10.3390/jpm12030435 ↗
17. Human-centered XAI: Developing design patterns for explanations of clinical decision support systems10.1016/j.ijhcs.2021.102684 ↗
18. A Clinical Decision Support System for Sleep Staging Tasks with Explanations from Artificial Intelligence: User-Centered Design and Evaluation Study10.2196/28659 ↗
19. Bussone, A., Stumpf, S., and O’Sullivan, D. (2015, January 21–23). The role of explanations on trust and reliance in clinical decision support systems. Proceedings of the 2015 international conference on healthcare informatics, Dallas, TX, USA.10.1109/ICHI.2015.26 ↗
20. Humans and automation: Use, misuse, disuse, abuse10.1518/001872097778543886 ↗
21. Wang, X., and Yin, M. (2021, January 14–17). Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. Proceedings of the 26th International Conference on Intelligent User Interfaces, College Station, TX, USA.10.1145/3397481.3450650 ↗
22. An explainable machine learning-based clinical decision support system for prediction of gestational diabetes mellitus10.1038/s41598-022-05112-2 ↗
23. Pregnancy exercise and nutrition with smartphone application support: A randomized controlled trial10.1097/AOG.0000000000002582 ↗
24. A unified approach to interpreting model predictions
25. Using a machine learning approach to predict mortality in critically ill influenza patients: A cross-sectional retrospective multicentre study in Taiwan10.1136/bmjopen-2019-033898 ↗
26. Taking advice: Accepting help, improving judgment, and sharing responsibility10.1006/obhd.1997.2697 ↗
27. Receiving other people’s advice: Influence and benefit10.1016/j.obhdp.2003.08.002 ↗
28. A case-based classifier for hypertension detection10.1016/j.knosys.2010.07.002 ↗
29. Cai, C.J., Jongejan, J., and Holbrook, J. (2019, January 17–20). The Effects of Example-Based Explanations in a Machine Learning Interface. Proceedings of the 24th International Conference on Intelligent User Interfaces, Marina del Ray, CA, USA.10.1145/3301275.3302289 ↗
30. Artificial intelligence explainability: The technical and ethical dimensions10.1098/rsta.2020.0363 ↗
31. Explaining Deep Learning using examples: Optimal feature weighting methods for twin systems using post-hoc, explanation-by-example in XAI10.1016/j.knosys.2021.107530 ↗
32. Caruana, R., Kangarloo, H., Dionisio, J.D., Sinha, U., and Johnson, D. (1999, January 6–10). Case-based explanation of non-case-based learning methods. Proceedings of the AMIA Symposium, American Medical Informatics Association, Washington, DC, USA.
33. Evaluating XAI: A comparison of rule-based and example-based explanations10.1016/j.artint.2020.103404 ↗
34. Matthews, M., Doherty, G., Coyle, D., and Sharry, J. (2008). Designing mobile applications to support mental health interventions. Handbook of Research on User Interface Design and Evaluation for Mobile Technology, IGI Global.10.4018/978-1-59904-871-0.ch038 ↗
35. Machine learning and medical education10.1038/s41746-018-0061-1 ↗