Reviewing CAM-Based Deep Explainable Methods in Healthcare
Source abstract
Abstract
The use of artificial intelligence within the healthcare sector is consistently growing. However, the majority of deep learning-based AI systems are of a black box nature, causing these systems to suffer from a lack of transparency and credibility. Due to the widespread adoption of medical imaging for diagnostic purposes, the healthcare industry frequently relies on methods that provide visual explanations, enhancing interpretability. Existing research has summarized and explored the usage of visual explanation methods in the healthcare domain, providing introductions to the methods that have been employed. However, existing reviews are frequently used for interpretable analysis in the medical field ignoring comprehensive reviews on Class Activation Mapping (CAM) methods because researchers typically categorize CAM under the broader umbrella of visual explanations without delving into specific applications in the healthcare sector. Therefore, this study primarily aims to analyze the specific applications of CAM-based deep explainable methods in the healthcare industry, following the PICO (Population, Intervention, Comparison, Outcome) framework. Specifically, we selected 45 articles for systematic review and comparative analysis from three databases—PubMed, Science Direct, and Web of Science—and then compared eight advanced CAM-based methods using five datasets to assist in method selection. Finally, we summarized current hotspots and future challenges in the application of CAM in the healthcare field.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Healthcare · Explainable Artificial Intelligence (XAI) · AI in cancer detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Software Engineering, Chengdu University of Information Technology, Chengdu 610225, China
Sichuan Province Engineering Technology Research Center of Support Software of Informatization Application, Chengdu 610225, China
Funding
Major special projects of science and Technology Department of Sichuan Province · Key R & D projects of Sichuan Science and Technology Department
References (75)
Show source-provided references
1. Convolutional neural networks for computer-aided detection or diagnosis in medical image analysis: An overview10.3934/mbe.2019326 ↗
2. Liang, X., Yu, J., Liao, J., and Chen, Z. (2020). Convolutional Neural Network for Breast and Thyroid Nodules Diagnosis in Ultrasound Imaging. BioMed Res. Int., 2020.10.1155/2020/1763803 ↗
3. Diabetic retinopathy classification based on multipath CNN and machine learning classifiers10.1007/s13246-021-01012-3 ↗
4. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2016, January 27–30). Learning Deep Features for Discriminative Localization. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.319 ↗
5. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22–29). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.74 ↗
6. Chattopadhay, A., Sarkar, A., Howlader, P., and Balasubramanian, V.N. (2018, January 12–15). Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.10.1109/WACV.2018.00097 ↗
7. Desai, S., and Ramaswamy, H.G. (2020, January 1–5). Ablation-CAM: Visual Explanations for Deep Convolutional Network via Gradient-free Localization. Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass, CO, USA.10.1109/WACV45572.2020.9093360 ↗
8. Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z., Ding, S., Mardziel, P., and Hu, X. (2020, January 14–19). Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.10.1109/CVPRW50498.2020.00020 ↗
9. A survey of visual analytics for Explainable Artificial Intelligence methods10.1016/j.cag.2021.09.002 ↗
10. Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13–17). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939778 ↗
11. Lundberg, S., and Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions. Adv. Neural Inf. Process. Syst., 30.
12. Loh, H.W., Ooi, C.P., Seoni, S., Barua, P.D., Molinari, F., and Acharya, U.R. (2022). Application of explainable artificial intelligence for healthcare: A systematic review of the last decade (2011–2022). Comput. Methods Programs Biomed., 226.10.1016/j.cmpb.2022.107161 ↗
13. A systematic review on the use of explainability in deep learning systems for computer aided diagnosis in radiology: Limited use of explainable AI?10.1016/j.ejrad.2022.110592 ↗
14. How does the model make predictions? A systematic literature review on the explainability power of machine learning in healthcare10.1016/j.artmed.2023.102616 ↗
15. Uses of Different Machine Learning Algorithms for Diagnosis of Dental Caries10.1155/2022/5032435 ↗
16. A compact and interpretable convolutional neural network for cross-subject driver drowsiness detection from single-channel EEG10.1016/j.ymeth.2021.04.017 ↗
17. Li, J., Huang, J., Jiang, T., Tu, L., Cui, L., Cui, J., Ma, X., Yao, X., Shi, Y., and Wang, S. (2022). A multi-step approach for tongue image classification in patients with diabetes. Comput. Biol. Med., 149.10.1016/j.compbiomed.2022.105935 ↗
18. Penso, M., Moccia, S., Caiani, E.G., Caredda, G., Lampus, M.L., Carerj, M.L., Babbaro, M., Pepi, M., Chiesa, M., and Pontone, G. (2023). A token-mixer architecture for CAD-RADS classification of coronary stenosis on multiplanar reconstruction CT images. Comput. Biol. Med., 153.10.1016/j.compbiomed.2022.106484 ↗
19. An Explainable 3D Residual Self-Attention Deep Neural Network for Joint Atrophy Localization and Alzheimer’s Disease Diagnosis Using Structural MRI10.1109/JBHI.2021.3066832 ↗
20. An Explainable AI driven Decision Support System for COVID-19 Diagnosis using Fused Classification and Segmentation
21. An Explainable Deep Learning Ensemble Model for Robust Diagnosis of Diabetic Retinopathy Grading10.1145/3469841 ↗
22. Oztekin, F., Katar, O., Sadak, F., Yildirim, M., Cakar, H., Aydogan, M., Ozpolat, Z., Talo Yildirim, T., Yildirim, O., and Faust, O. (2023). An Explainable Deep Learning Model to Prediction Dental Caries Using Panoramic Radiograph Images. Diagnostics, 13.10.3390/diagnostics13020226 ↗
23. An explainable self-attention deep neural network for detecting mild cognitive impairment using multi-input digital drawing tasks10.1186/s13195-022-01043-2 ↗
24. An interpretable decision-support model for breast cancer diagnosis using histopathology images10.1016/j.jpi.2023.100319 ↗
25. Automated detection of scaphoid fractures using deep neural networks in radiographs10.1016/j.engappai.2023.106165 ↗
26. Automatic detection of tympanic membrane and middle ear infection from oto-endoscopic images via convolutional neural networks10.1016/j.neunet.2020.03.023 ↗
27. Automatic morphological classification of mitral valve diseases in echocardiographic images based on explainable deep learning methods10.1007/s11548-021-02542-7 ↗
28. Automatic Sleep Stage Classification Using Deep Learning Algorithm for Multi-Institutional Database10.1109/ACCESS.2023.3275087 ↗
29. Cerebral hemorrhage detection and localization with medical imaging for cerebrovascular disease diagnosis and treatment using explainable deep learning10.1007/s40042-021-00202-2 ↗
30. Convolutional neural network for automated classification of osteonecrosis and related mandibular trabecular patterns10.1016/j.bonr.2022.101632 ↗
31. COVID-19 Classification from Chest X-ray Images: A Framework of Deep Explainable Artificial Intelligence10.1155/2022/4254631 ↗
32. COVID-19 classification using chest X-ray images based on fusion-assisted deep Bayesian optimization and Grad-CAM visualization10.3389/fpubh.2022.1046296 ↗
33. Suri, J.S., Agarwal, S., Chabert, G.L., Carriero, A., Pasche, A., Danna, P.S.C., Saba, L., Mehmedovic, A., Faa, G., and Singh, I.M. (2022). COVLIAS 2.0-cXAI: Cloud-Based Explainable Deep Learning System for COVID-19 Lesion Localization in Computed Tomography Scans. Diagnostics, 12.10.3390/diagnostics12061482 ↗
34. DeepClassPathway: Molecular pathway aware classification using explainable deep learning10.1016/j.ejca.2022.08.033 ↗
35. Toğaçar, M., Muzoğlu, N., Ergen, B., Yarman, B.S.B., and Halefoğlu, A.M. (2022). Detection of COVID-19 findings by the local interpretable model-agnostic explanations method of types-based activations extracted from CNNs. Biomed. Signal Process. Control, 71.10.1016/j.bspc.2021.103128 ↗
36. DIAG a Diagnostic Web Application Based on Lung CT Scan Images and Deep Learning
37. Yang, M., Huang, X., Huang, L., and Cai, G. (2023). Diagnosis of Parkinson’s disease based on 3D ResNet: The frontal lobe is crucial. Biomed. Signal Process. Control, 85.10.1016/j.bspc.2023.104904 ↗
38. EEG-based intention recognition with deep recurrent-convolution neural network: Performance and channel selection by Grad-CAM10.1016/j.neucom.2020.07.072 ↗
39. Electrocardiogram analysis of post-stroke elderly people using one-dimensional convolutional neural network model with gradient-weighted class activation mapping10.1016/j.artmed.2022.102342 ↗
40. Mukhtorov, D., Rakhmonova, M., Muksimova, S., and Cho, Y.-I. (2023). Endoscopic Image Classification Based on Explainable Deep Learning. Sensors, 23.10.3390/s23063176 ↗
41. Explainable Artificial Intelligence Model for Diagnosis of Atrial Fibrillation Using Holter Electrocardiogram Waveforms10.1536/ihj.21-094 ↗
42. Explainable Deep Learning-Based Approach for Multilabel Classification of Electrocardiogram
43. Jahmunah, V., Ng, E.Y.K., Tan, R.-S., Oh, S.L., and Acharya, U.R. (2022). Explainable detection of myocardial infarction using deep learning models with Grad-CAM technique on ECG signals. Comput. Biol. Med., 146.10.1016/j.compbiomed.2022.105550 ↗
44. Explainable end-to-end deep learning for diabetic retinopathy detection across multiple datasets10.1117/1.JMI.7.4.044503 ↗
45. Explainable framework for Glaucoma diagnosis by image processing and convolutional neural network synergy: Analysis with doctor evaluation10.1016/j.future.2021.11.018 ↗
46. Explainable multiple abnormality classification of chest CT volumes10.1016/j.artmed.2022.102372 ↗
47. Islam, R., Goni, O.F., Sayeed, A., Anower, S., Ahsan, M., and Haider, J. (2022). Explainable Transformer-Based Deep Learning Model for the Detection of Malaria Parasites from Blood Cell Images. Sensors, 22.10.3390/s22124358 ↗
48. Hossain, S.I., Herve, J.d.G.d., Hassan, M.S., Martineau, D., Petrosyan, E., Corbin, V., Beytout, J., Lebert, I., Durand, J., and Carravieri, I. (2022). Exploring convolutional neural networks with transfer learning for diagnosing Lyme disease from skin lesion images. Comput. Methods Programs Biomed., 215.10.1016/j.cmpb.2022.106624 ↗
49. Interpretation and Classification of Arrhythmia Using Deep Convolutional Network10.1109/TIM.2022.3204316 ↗
50. Choi, Y., and Lee, H. (2023). Interpretation of lung disease classification with light attention connected module. Biomed. Signal Process. Control, 84.10.1016/j.bspc.2023.104695 ↗
51. Intracerebral hemorrhage detection on computed tomography images using a residual neural network10.1016/j.ejmp.2022.05.015 ↗
52. Dabass, M., Vashisth, S., and Vig, R. (2022). MTU: A multi-tasking U-net with hybrid convolutional learning and attention modules for cancer classification and gland Segmentation in Colon Histopathological Images. Comput. Biol. Med., 150.10.1016/j.compbiomed.2022.106095 ↗
53. Afify, H.M., Mohammed, K.K., and Ella Hassanien, A. (2023). Novel prediction model on OSCC histopathological images via deep transfer learning combined with Grad-CAM interpretation. Biomed. Signal Process. Control, 83.10.1016/j.bspc.2023.104704 ↗
54. Sunija, A.P., Kar, S., Gayathri, S., Gopi, V.P., and Palanisamy, P. (2021). OctNET: A Lightweight CNN for Retinal Disease Classification from Optical Coherence Tomography Images. Comput. Methods Programs Biomed., 200.10.1016/j.cmpb.2020.105877 ↗
55. Parkinson’s disease is characterized by sub-second resting-state spatio-oscillatory patterns: A contribution from deep convolutional neural network10.1016/j.nicl.2022.103266 ↗
56. Li, H., Dong, X., Shen, W., Ge, F., and Li, H. (2022). Resampling-based cost loss attention network for explainable imbalanced diabetic retinopathy grading. Comput. Biol. Med., 149.10.1016/j.compbiomed.2022.105970 ↗
57. SSPNet: An interpretable 3D-CNN for classification of schizophrenia using phase maps of resting-state complex-valued fMRI data10.1016/j.media.2022.102430 ↗
58. Texture analysis for muscular dystrophy classification in MRI with improved class activation mapping10.1016/j.patcog.2018.08.012 ↗
59. Toward understanding deep learning classification of anatomic sites: Lessons from the development of a CBCT projection classifier
60. Visualization deep learning model for automatic arrhythmias classification10.1088/1361-6579/ac8469 ↗
61. He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.10.1109/CVPR.2016.90 ↗
62. Tan, M., and Le, Q.V. (2020). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. arXiv.
63. The Pascal Visual Object Classes (VOC) Challenge10.1007/s11263-009-0275-4 ↗
64. ROC analysis of classifiers in machine learning: A survey10.3233/IDA-130592 ↗
65. A systematic analysis of performance measures for classification tasks10.1016/j.ipm.2009.03.002 ↗
66. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions10.1038/sdata.2018.161 ↗
67. Dataset of breast ultrasound images10.1016/j.dib.2019.104863 ↗
68. Brain tumor classification based on neural architecture search10.1038/s41598-022-22172-6 ↗
69. Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning10.1016/j.cell.2018.02.010 ↗
70. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2014). Going Deeper with Convolutions. arXiv.10.1109/CVPR.2015.7298594 ↗
71. Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2018). Densely Connected Convolutional Networks. arXiv.10.1109/CVPR.2017.243 ↗
72. Fu, R., Hu, Q., Dong, X., Guo, Y., Gao, Y., and Li, B. (2020). Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs. arXiv.
73. Muhammad, M.B., and Yeasin, M. (2020, January 19–24). Eigen-CAM: Class Activation Map using Principal Components. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.10.1109/IJCNN48605.2020.9206626 ↗
74. LayerCAM: Exploring Hierarchical Class Activation Maps for Localization10.1109/TIP.2021.3089943 ↗
75. Srinivas, S., and Fleuret, F. (2019). Full-Gradient Representation for Neural Network Visualization. arXiv.