Transparency of Artificial Intelligence in Healthcare: Insights from Professionals in Computing and Healthcare Worldwide
Source abstract
Abstract
Although it is widely assumed that Artificial Intelligence (AI) will revolutionise healthcare in the near future, considerable progress must yet be made in order to gain the trust of healthcare professionals and patients. Improving AI transparency is a promising avenue for addressing such trust issues. However, transparency still lacks maturation and definitions. We seek to answer what challenges do experts and professionals in computing and healthcare identify concerning transparency of AI in healthcare? Here, we examine AI transparency in healthcare from five angles: interpretability, privacy, security, equity, and intellectual property. We respond to this question based on recent literature discussing the transparency of AI in healthcare and on an international online survey we sent to professionals working in computing and healthcare and potentially within AI. We collected responses from 40 professionals around the world. Overall, the survey results and current state of the art suggest key problems are a generalised lack of information available to the general public, a lack of understanding of transparency aspects covered in this work, and a lack of involvement of all stakeholders in the development of AI systems. We propose a set of recommendations, the implementation of which can enhance the transparency of AI in healthcare.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Artificial Intelligence in Healthcare and Education · Ethics and Social Impacts of AI · Privacy-Preserving Technologies in Data
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
German Center for Neurodegenerative Diseases (DZNE), 39120 Magdeburg, Germany
Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke University Magdeburg, 39120 Magdeburg, Germany
Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh EH16 4SB, UK
DCU School of Computing, Dublin City University, D09 DXA0 Dublin, Ireland
References (89)
Show source-provided references
1. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: A systematic review and meta-analysis10.1016/S2589-7500(19)30123-2 ↗
2. Bacciu, D., Lisboa, P.J.G., and Vellido, A. (2021). Deep learning for medical imaging. Deep Learning in Biology and Medicine, World Scientific (Europe).10.1142/q0322 ↗
3. Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: A review10.1016/j.artmed.2018.08.008 ↗
4. The potential for artificial intelligence in healthcare10.7861/futurehosp.6-2-94 ↗
5. Mazo, C., Kearns, C., Mooney, C., and Gallagher, W.M. (2020). Clinical Decision Support Systems in Breast Cancer: A Systematic Review. Cancers, 12.10.3390/cancers12020369 ↗
6. Review of Machine Learning in Predicting Dermatological Outcomes10.3389/fmed.2020.00266 ↗
7. A review on current advances in machine learning based diabetes prediction10.1016/j.pcd.2021.02.005 ↗
8. Vellameeran, F.A., and Brindha, T. (2021). An integrated review on machine learning approaches for heart disease prediction: Direction towards future research gaps. Bio-Algorithms-Med-Syst., 20200069.10.1515/bams-2020-0069 ↗
9. Using machine-learning risk prediction models to triage the acuity of undifferentiated patients entering the emergency care system: A systematic review
10. Machine learning for genetic prediction of psychiatric disorders: A systematic review10.1038/s41380-020-0825-2 ↗
11. An explainable machine learning-based clinical decision support system for prediction of gestational diabetes mellitus
12. Machine learning approaches in COVID-19 diagnosis, mortality, and severity risk prediction: A review10.1016/j.imu.2021.100564 ↗
13. Applications of machine learning to diagnosis and treatment of neurodegenerative diseases10.1038/s41582-020-0377-8 ↗
14. A Comprehensive Review on Medical Diagnosis Using Machine Learning
15. Systematic review of machine learning for diagnosis and prognosis in dermatology10.1080/09546634.2019.1682500 ↗
16. Machine Learning and Intelligent Diagnostics in Dental and Orofacial Pain Management: A Systematic Review
17. Borchert, R., Azevedo, T., Badhwar, A., Bernal, J., Betts, M., Bruffaerts, R., Burkhart, M., Dewachter, I., Gellersen, H., and Low, A. (2021). Artificial intelligence for diagnosis and prognosis in neuroimaging for dementia; a systematic review. medRxiv.10.1101/2021.12.12.21267677 ↗
18. A scoping review of machine learning in psychotherapy research10.1080/10503307.2020.1808729 ↗
19. Sajjadian, M., Lam, R.W., Milev, R., Rotzinger, S., Frey, B.N., Soares, C.N., Parikh, S.V., Foster, J.A., Turecki, G., and Müller, D.J. (2021). Machine learning in the prediction of depression treatment outcomes: A systematic review and meta-analysis. Psychol. Med., 1–10.10.1017/S0033291721003871 ↗
20. Future of biomarker evaluation in the realm of artificial intelligence algorithms: Application in improved therapeutic stratification of patients with breast and prostate cancer10.1136/jclinpath-2020-207351 ↗
21. Artificial Intelligence–Assisted System in Postoperative Follow-up of Orthopedic Patients: Exploratory Quantitative and Qualitative Study10.2196/16896 ↗
22. How artificial intelligence could transform emergency department operations10.1016/j.ajem.2018.01.017 ↗
23. Del-Valle-Soto, C., Nolazco-Flores, J.A., Puerto-Flores, D., Alberto, J., Velázquez, R., Valdivia, L.J., Rosas-Caro, J., and Visconti, P. (2022). Statistical Study of User Perception of Smart Homes during Vital Signal Monitoring with an Energy-Saving Algorithm. Int. J. Environ. Res. Public Health, 19.10.3390/ijerph19169966 ↗
24. Artificial intelligence in healthcare10.1038/s41551-018-0305-z ↗
25. Ethical challenges regarding artificial intelligence in medicine from the perspective of scientific editing and peer review10.6087/kcse.164 ↗
26. Bohr, A., and Memarzadeh, K. (2020). Ethical and Legal Challenges of Artificial Intelligence-Driven Health Care. Artificial Intelligence in Healthcare, Elsevier. [1st ed.].10.1016/B978-0-12-818438-7.00002-2 ↗
27. WHO (2021). Health Ethics & Governance of Artificial Intelligence for Health, World Health Organization.
28. A healthy debate: Exploring the views of medical doctors on the ethics of artificial intelligence10.1016/j.artmed.2021.102190 ↗
29. Artificial intelligence in healthcare: A critical analysis of the legal and ethical implications10.1093/ijlit/eaz004 ↗
30. Application of Artificial Intelligence in Healthcare: Chances and Challenges10.9734/cjast/2021/v40i631320 ↗
31. Mazo, C., Aura, C., Rahman, A., Gallagher, W.M., and Mooney, C. (2022). Application of Artificial Intelligence Techniques to Predict Risk of Recurrence of Breast Cancer: A Systematic Review. J. Pers. Med., 12.10.3390/jpm12091496 ↗
32. Towards Transparency by Design for Artificial Intelligence10.1007/s11948-020-00276-4 ↗
33. Weller, A. (2019). Transparency: Motivations and challenges. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer.10.1007/978-3-030-28954-6_2 ↗
34. The WU-Minn human connectome project: An overview10.1016/j.neuroimage.2013.05.041 ↗
35. Use of AI-based tools for healthcare purposes: A survey study from consumers’ perspectives10.1186/s12911-020-01191-1 ↗
36. Artificial intelligence in cancer research, diagnosis and therapy10.1038/s41568-021-00399-1 ↗
37. Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J.M.F., and Eckersley, P. (2019). Explainable Machine Learning in Deployment. arXiv.10.1145/3351095.3375624 ↗
38. Definitions, methods, and applications in interpretable machine learning10.1073/pnas.1900654116 ↗
39. Antoniadi, A.M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B.A., and Mooney, C. (2021). Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review. Appl. Sci., 11.10.3390/app11115088 ↗
40. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)10.1109/ACCESS.2018.2870052 ↗
41. Algo-Rhythms and the Beat of the Legal Drum10.1007/s13347-017-0277-z ↗
42. Bartneck, C., Lütge, C., Wagner, A., and Welsh, S. (2021). Privacy Issues of AI, Springer International Publishing.10.1007/978-3-030-51110-4_8 ↗
43. USA (2021, November 08). Health Insurance Portability and Accountability Act of 1996, Available online: https://www.govinfo.gov/content/pkg/PLAW-104publ191/pdf/PLAW-104publ191.pdf.
44. Johnson, L. (2016). Chapter 3—Statutory and Regulatory GRC. Security Controls Evaluation, Testing, and Assessment Handbook, Syngress.10.1016/B978-0-12-802324-2.00001-4 ↗
45. FEDERAL-TRADE-COMMISSION (2021, November 08). Children’s Online Privacy Protection Rule, Available online: https://www.ftc.gov/system/files/2012-31341.pdf.
46. EU (2021, October 27). General Data Protection Regulation. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679.
47. Population Bias in polygenic risk prediction models for coronary artery disease10.1161/CIRCGEN.120.002932 ↗
48. Noor, P. (2020). Can we trust AI not to further embed racial bias and prejudice?. BMJ, 368.10.1136/bmj.m363 ↗
49. Reducing racial bias in AI models for clinical use requires a top-down intervention10.1038/s42256-021-00362-7 ↗
50. Addressing bias in artificial intelligence in health care10.1001/jama.2019.18058 ↗
51. WIPO (2020). What Is Intellectual Property?, World Intellectual Property Organization (WIPO).
52. Survey Fatigue During the COVID-19 Pandemic: An Analysis of Neurosurgery Survey Response Rates10.3389/fsurg.2021.690680 ↗
53. COVID-19 Survey Burden for Healthcare Workers: Literature Review and Audit10.1016/j.puhe.2021.05.006 ↗
54. Improving the Quality of Web Surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES)10.2196/jmir.6.3.e34 ↗
55. Artificial intelligence and its impact on the domains of Universal Health Coverage, Health Emergencies and Health Promotion: An overview of systematic reviews10.1016/j.ijmedinf.2022.104855 ↗
56. The importance of interpretability and visualization in machine learning for applications in medicine and health care10.1007/s00521-019-04051-w ↗
57. Price, I., and Nicholson, W. (2017). Artificial intelligence in health care: Applications and legal issues. SciTech. Lawyer, 10, Available online: https://ssrn.com/abstract=3078704.
58. Patient apprehensions about the use of artificial intelligence in healthcare10.1038/s41746-021-00509-1 ↗
59. Interpretability in healthcare: A comparative study of local machine learning interpretability techniques10.1111/coin.12410 ↗
60. Interpretability of Machine Learning Solutions in Public Healthcare: The CRISP-ML Approach10.3389/fdata.2021.660206 ↗
61. Explainability for artificial intelligence in healthcare: A multidisciplinary perspective10.1186/s12911-020-01332-6 ↗
62. Stiglic, G., Kocbek, P., Fijacko, N., Zitnik, M., Verbert, K., and Cilar, L. (2020). Interpretability of machine learning based prediction models in healthcare. arXiv.10.1002/widm.1379 ↗
63. Das, A., and Rad, P. (2020). Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey. arXiv.
64. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI10.1016/j.inffus.2019.12.012 ↗
65. The need for uncertainty quantification in machine-assisted medical decision making10.1038/s42256-018-0004-1 ↗
66. Interpretable Artificial Intelligence: Why and When10.2214/AJR.19.22145 ↗
67. Zhou, J., Gandomi, A.H., Chen, F., and Holzinger, A. (2021). Evaluating the quality of machine learning explanations: A survey on methods and metrics. Electronics, 10.10.3390/electronics10050593 ↗
68. Availability of health data: Requirements and solutions10.1016/S1386-5056(98)00016-1 ↗
69. Benefits and challenges of Big Data in healthcare: An overview of the European initiatives10.1093/eurpub/ckz168 ↗
70. Bartoletti, I. (2019, January 26–29). AI in healthcare: Ethical and privacy challenges. Proceedings of the Conference on Artificial Intelligence in Medicine in Europe, Poznan, Poland.10.1007/978-3-030-21642-9_2 ↗
71. Georgiou, D., and Lambrinoudakis, C. (2020). Compatibility of a Security Policy for a Cloud-Based Healthcare System with the EU General Data Protection Regulation (GDPR). Information, 11.10.20944/preprints202010.0577.v1 ↗
72. Seh, A.H., Zarour, M., Alenezi, M., Sarkar, A.K., Agrawal, A., Kumar, R., and Ahmad Khan, R. (2020). Healthcare data breaches: Insights and implications. Healthcare, 8.10.3390/healthcare8020133 ↗
73. Healthcare data breaches: Implications for digital forensic readiness10.1007/s10916-018-1123-2 ↗
74. Shavers, B., and Bair, J. (2016). Chapter 6—Cryptography and Encryption. Hiding Behind the Keyboard, Syngress.10.1016/B978-0-12-803340-1.00006-9 ↗
75. The eu general data protection regulation (gdpr)
76. Database Security: An Overview and Analysis of Current Trend
77. Regulation, G.D.P. (2018). General data protection regulation (GDPR). Intersoft Consult. Accessed Oct., 24.
78. The ethics of algorithms: Key problems and solutions10.1007/s00146-021-01154-8 ↗
79. Pulse oximetry, racial bias and statistical bias10.1186/s13613-021-00974-7 ↗
80. Federated learning: Challenges, methods, and future directions
81. Challenges to transparency involving intellectual property and privacy concerns in life cycle assessment/costing: A case study of new flame retarded polymers10.1016/j.cesys.2021.100045 ↗
82. WHO (2022, January 24). Operationalising the COVID-19 Technology Access Pool (C-TAP). Available online: https://www.who.int/publications/m/item/c-tap-a-concept-paper.
83. Preserving intellectual property rights: Managerial insight into the escalating counterfeit market quandary10.1016/j.bushor.2008.04.003 ↗
84. Kim, H.L., Hovav, A., and Han, J. (2020). Protecting intellectual property from insider threats: A management information security intelligence perspective. J. Intellect. Cap., Available online: https://www.emerald.com/insight/content/doi/10.1108/JIC-05-2019-0096/full/html.10.1108/JIC-05-2019-0096 ↗
85. Halt, G.B., Donch, J.C., Stiles, A.R., VanLuvanee, L.J., Theiss, B.R., and Blue, D.L. (2019). Tips for Avoiding and Preventing Intellectual Property Problems. FDA and Intellectual Property Strategies for Medical Device Technologies, Springer.10.1007/978-3-030-04462-6 ↗
86. EU4Health-Programme (2021, October 27). eHealth Action Plan 2021–2027. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=OJ:L:2021:107:FULL&from=EN.
87. ACM (2021, October 27). US Technology Policy Committee. Available online: https://www.acm.org/binaries/content/assets/public-policy/ustpc-comments-fda-software-based-device-safety-060319.pdf.
88. ACM (2021, October 27). Statement on Algorithmic Transparency and Accountability. Available online: https://iapp.org/media/pdf/resource_center/2017_usacm_statement_algorithms.pdf.
89. The role of telehealth during COVID-19 outbreak: A systematic review based on current evidence10.1186/s12889-020-09301-4 ↗