Seg-Eigen-CAM: Eigen-Value-Based Visual Explanations for Semantic Segmentation Models
Source abstract
Abstract
In recent years, most Explainable Artificial Intelligence methods have primarily focused on image classification. Although research on interpretability in image segmentation has been increasing, it remains relatively limited. As an extension of Grad-CAM, several methods have been proposed and applied to image segmentation with the aim of enhancing existing techniques and adapting their properties. However, in this study, we highlight a common issue with gradient-based methods when generating visual explanations—these methods tend to emphasize background information, resulting in significant noise, especially when dealing with image segmentation tasks involving complex or cluttered backgrounds. Inspired by the widely used Eigen-CAM method, this study proposes a novel explainability approach tailored for semantic segmentation. By integrating gradient information and introducing a sign correction strategy, our method enhances spatial localization and reduces background noise, particularly in complex scenes. Through empirical studies, we compare our method with several representative methods, employing multiple evaluation metrics to quantify explainability and validate the advantages of our method. Overall, this study advances explainability methods for convolutional neural networks in semantic segmentation. Our approach not only preserves localized attention but also offers a simpler and more intuitive CAM, which has the potential to play a crucial role in sensitive application scenarios, fostering the development of trustworthy AI models.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Explainable Artificial Intelligence (XAI) · Advanced Neural Network Applications · Multimodal Machine Learning Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Management Information Systems, National Chung Hsing University, Taichung 402, Taiwan
Funding
Smart Sustainable New Agriculture Research Center (SMARTer)
References (38)
Show source-provided references
1. Imagenet classification with deep convolutional neural networks
2. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
3. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
4. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
5. Shrikumar, A., Greenside, P., Shcherbina, A., and Kundaje, A. (2016). Not just a black box: Learning important features through propagating activation differences. arXiv.
6. Binder, A., Montavon, G., Lapuschkin, S., Müller, K.R., and Samek, W. (2016, January 6–9). Layer-wise relevance propagation for neural networks with local renormalization layers. Proceedings of the Artificial Neural Networks and Machine Learning–ICANN 2016: 25th International Conference on Artificial Neural Networks, Barcelona, Spain. Proceedings, Part II.10.1007/978-3-319-44781-0_8 ↗
7. Zeiler, M.D., and Fergus, R. (2014, January 6–12). Visualizing and understanding convolutional networks. Proceedings of the Computer Vision—ECCV 2014: 13th European Conference, Zurich, Switzerland. Proceedings, Part I.10.1007/978-3-319-10590-1_53 ↗
8. Springenberg, J.T., Dosovitskiy, A., Brox, T., and Riedmiller, M. (2014). Striving for simplicity: The all convolutional net. arXiv.
9. Sundararajan, M., Taly, A., and Yan, Q. (2017, January 6–11). Axiomatic attribution for deep networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia. PMLR.
10. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2016, January 27–30). Learning deep features for discriminative localization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.319 ↗
11. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22–29). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.74 ↗
12. Chattopadhay, A., Sarkar, A., Howlader, P., and Balasubramanian, V.N. (2018, January 12–15). Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.10.1109/WACV.2018.00097 ↗
13. Layercam: Exploring hierarchical class activation maps for localization10.1109/TIP.2021.3089943 ↗
14. Fu, R., Hu, Q., Dong, X., Guo, Y., Gao, Y., and Li, B. (2020). Axiom-based grad-cam: Towards accurate visualization and explanation of cnns. arXiv.10.5244/C.34.146 ↗
15. Muhammad, M.B., and Yeasin, M. (2020, January 19–24). Eigen-cam: Class activation map using principal components. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.10.1109/IJCNN48605.2020.9206626 ↗
16. Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z., Ding, S., Mardziel, P., and Hu, X. (2020, January 14–19). Score-CAM: Score-weighted visual explanations for convolutional neural networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00020 ↗
17. Ramaswamy, H.G. (2020, January 1–5). Ablation-cam: Visual explanations for deep convolutional network via gradient-free localization. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2020, Snowmass Village, CO, USA.
18. Vinogradova, K., Dibrov, A., and Myers, G. (2020, January 7–12). Towards interpretable semantic segmentation via gradient-weighted class activation mapping (student abstract). Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i10.7244 ↗
19. Hasany, S.N., Petitjean, C., and Mériaudeau, F. (2023, January 17–24). Seg-xres-cam: Explaining spatially local regions in image segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023, Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00384 ↗
20. Draelos, R.L., and Carin, L. (2020). Use HiResCAM instead of Grad-CAM for faithful explanations of convolutional neural networks. arXiv.
21. Golub, G.H., and Van Loan, C.F. (2013). Matrix Computations, JHU Press.10.56021/9781421407944 ↗
22. Higham, N.J. (2020). Accuracy and Stability of Numerical Algorithms, Society for Industrial and Applied Mathematics.
23. Learning the parts of objects by non-negative matrix factorization10.1038/44565 ↗
24. Resolving the sign ambiguity in the singular value decomposition
25. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany. Proceedings, Part III.10.1007/978-3-319-24574-4_28 ↗
26. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
27. Chen, L.C. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv.10.1007/978-3-030-01234-2_49 ↗
28. Dugăeșescu, A., and Florea, A.M. (2022, January 12–15). Evaluation of Class Activation Methods for Understanding Image Classification Tasks. Proceedings of the 2022 24th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), Linz, Austria.10.1109/SYNASC57785.2022.00033 ↗
29. Dardouillet, P., Benoit, A., Amri, E., Bolon, P., Dubucq, D., and Crédoz, A. (2022). Explainability of image semantic segmentation through shap values. International Conference on Pattern Recognition, Springer Nature.10.1007/978-3-031-37731-0_19 ↗
30. Visual attribution for deep learning segmentation in medical imaging
31. Gizzini, A.K., Shukor, M., and Ghandour, A.J. (2023). Extending cam-based xai methods for remote sensing imagery segmentation. arXiv.
32. A mathematical theory of communication10.1002/j.1538-7305.1948.tb01338.x ↗
33. Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., and Müller, K.R. (2019). Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer Nature.10.1007/978-3-030-28954-6 ↗
34. Wang, Y., Zhang, T., Guo, X., and Shen, Z. (2024). Gradient based Feature Attribution in Explainable AI: A Technical Review. arXiv.
35. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland. Proceedings, Part V.10.1007/978-3-319-10602-1_48 ↗
36. Contributors, M. (2020). MMSegmentation: Openmmlab Semantic Segmentation Toolbox and Benchmark, Github.
37. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
38. Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., and Sang, N. (2018, January 8–14). Bisenet: Bilateral segmentation network for real-time semantic segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01261-8_20 ↗