Data Augmentation in Histopathological Classification: An Analysis Exploring GANs with XAI and Vision Transformers
Source abstract
Abstract
Generative adversarial networks (GANs) create images by pitting a generator (G) against a discriminator (D) network, aiming to find a balance between the networks. However, achieving this balance is difficult because G is trained based on just one value representing D’s prediction, and only D can access image features. We introduce a novel approach for training GANs using explainable artificial intelligence (XAI) to enhance the quality and diversity of generated images in histopathological datasets. We leverage XAI to extract feature information from D and incorporate it into G via the loss function, a unique strategy not previously explored in this context. We demonstrate that this approach enriches the training with relevant information and promotes improved quality and more variability in the artificial images, decreasing the FID by up to 32.7% compared to traditional methods. In the data augmentation task, these images improve the classification accuracy of Transformer models by up to 3.81% compared to models without data augmentation and up to 3.01% compared to traditional GAN data augmentation. The Saliency method provides G with the most informative feature information. Overall, our work highlights the potential of XAI for enhancing GAN training and suggests avenues for further exploration in this field.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: AI in cancer detection · Generative Adversarial Networks and Image Synthesis · Cell Image Analysis Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Statistics (DCCE), São Paulo State University (UNESP), Rua Cristóvão Colombo, 2265, São José do Rio Preto 15054-000, SP, Brazil
Department of Computer Science and Engineering (DISI), University of Bologna, Via dell’ Università, 50, 47522 Cesena, Italy
Science and Technology Institute, Federal University of São Paulo (UNIFESP), Avenida Cesare Mansueto Giulio Lattes, 1201, São José dos Campos 12247-014, SP, Brazil
Faculty of Computer Science (FACOM), Federal University of Uberlândia (UFU), Avenida João Naves de Ávila, 2121, Bl.B, Uberlândia 38400-902, MG, Brazil
Funding
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brasil (CAPES) · National Council for Scientific and Technological Development CNPq · State of Minas Gerais Research Foundation—FAPEMIG · São Paulo Research Foundation—FAPESP
References (39)
Show source-provided references
1. Representation Learning: A Review and New Perspectives10.1109/TPAMI.2013.50 ↗
2. A survey of deep learning and its applications: A new paradigm to machine learning10.1007/s11831-019-09344-w ↗
3. Madani, A., Moradi, M., Karargyris, A., and Syeda-Mahmood, T. (2018, January 4–7). Semi-supervised learning with generative adversarial networks for chest X-ray classification with ability of data domain adaptation. Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA.10.1109/ISBI.2018.8363749 ↗
4. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification10.1016/j.neucom.2018.09.013 ↗
5. Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., and Weinberger, K. (2014). Generative Adversarial Nets. Advances in Neural Information Processing Systems, Curran Associates, Inc.
6. A survey on image data augmentation for deep learning10.1186/s40537-019-0197-0 ↗
7. Equilibrium points in n-person games10.1073/pnas.36.1.48 ↗
8. A review on Generative Adversarial Networks for image generation10.1016/j.cag.2023.05.010 ↗
9. Wang, J., Yang, C., Xu, Y., Shen, Y., Li, H., and Zhou, B. (2022, January 18–24). Improving gan equilibrium by raising spatial awareness. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01100 ↗
10. Bai, Q., Yang, C., Xu, Y., Liu, X., Yang, Y., and Shen, Y. (2023, January 17–22). Glead: Improving gans with a generator-leading task. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Washington, DC, USA.10.1109/CVPR52729.2023.01164 ↗
11. Wang, Z., She, Q., and Ward, T.E. (2021). Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy. ACM Comput. Surv., 54.10.1145/3439723 ↗
12. Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). Improved Training of Wasserstein GANs. Advances in Neural Information Processing Systems, Curran Associates, Inc.
13. Jolicoeur-Martineau, A. (2019, January 1–4). The relativistic discriminator: A key element missing from standard GAN. Proceedings of the International Conference on Learning Representations, Nice, France.
14. Robust Explainability: A tutorial on gradient-based attribution methods for deep neural networks10.1109/MSP.2022.3142719 ↗
15. Simonyan, K., Vedaldi, A., and Zisserman, A. (2014, January 14–16). Deep inside convolutional networks: Visualising image classification models and saliency maps. Proceedings of the International Conference on Learning Representations (ICLR), Banff, AB, Canada.
16. Precup, D., and Teh, Y.W. (2017, January 6–11). Learning Important Features Through Propagating Activation Differences. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia. PMLR, Proceedings of Machine Learning Research.
17. Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13–17). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA. KDD ’16.10.1145/2939672.2939778 ↗
18. Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems, Curran Associates, Inc.
19. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 3–7). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Proceedings of the International Conference on Learning Representations, Virtual.
20. Wang, W., Xie, E., Li, X., Fan, D.P., Song, K., Liang, D., Lu, T., Luo, P., and Shao, L. (2021, January 11–17). Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00061 ↗
21. Meila, M., and Zhang, T. (2021, January 18–24). Training data-efficient image transformers & distillation through attention. Proceedings of the 38th International Conference on Machine Learning, Virtual. PMLR, Proceedings of Machine Learning Research.
22. CoAtNet: Marrying Convolution and Attention for All Data Sizes
23. Maurício, J., Domingues, I., and Bernardino, J. (2023). Comparing Vision Transformers and Convolutional Neural Networks for Image Classification: A Literature Review. Appl. Sci., 13.10.3390/app13095521 ↗
24. Vision Transformers for Computational Histopathology10.1109/RBME.2023.3297604 ↗
25. Kernel Attention Transformer for Histopathology Whole Slide Image Analysis and Assistant Cancer Diagnosis10.1109/TMI.2023.3264781 ↗
26. Atabansi, C.C., Nie, J., Liu, H., Song, Q., Yan, L., and Zhou, X. (2023). A survey of Transformer applications for histopathological image analysis: New developments and future directions. Biomed. Eng. Online, 22.10.1186/s12938-023-01157-0 ↗
27. Baroni, G.L., Rasotto, L., Roitero, K., Tulisso, A., Di Loreto, C., and Della Mea, V. (2024). Optimizing Vision Transformers for Histopathology: Pretraining and Normalization in Breast Cancer Classification. J. Imaging, 10.10.3390/jimaging10050108 ↗
28. Vision transformer based classification of gliomas from histopathological images10.1016/j.eswa.2023.122672 ↗
29. A novel convolution transformer-based network for histopathology-image classification using adaptive convolution and dynamic attention10.1016/j.engappai.2024.108824 ↗
30. Selective synthetic augmentation with HistoGAN for improved histopathology image classification10.1016/j.media.2020.101816 ↗
31. Data augmentation guided breast cancer diagnosis and prognosis using an integrated deep-generative framework based on breast tumor’s morphological information10.1016/j.imu.2023.101171 ↗
32. A Data Augmentation Methodology to Reduce the Class Imbalance in Histopathology Images10.1007/s10278-024-01018-9 ↗
33. Brancati, N., and Frucci, M. (2024). Improving Breast Tumor Multi-Classification from High-Resolution Histological Images with the Integration of Feature Space Data Augmentation. Information, 15.10.3390/info15020098 ↗
34. Ruiz-Casado, J.L., Molina-Cabello, M.A., and Luque-Baena, R.M. (2024). Enhancing Histopathological Image Classification Performance through Synthetic Data Generation with Generative Adversarial Networks. Sensors, 24.10.20944/preprints202405.0467.v1 ↗
35. Gland segmentation in colon histology images: The glas challenge contest10.1016/j.media.2016.08.008 ↗
36. IICBU 2008: A proposed benchmark suite for biological image analysis10.1007/s11517-008-0380-5 ↗
37. Drelie Gelasca, E., Byun, J., Obara, B., and Manjunath, B. (2008, January 12–15). Evaluation and benchmark for biological image segmentation. Proceedings of the 2008 15th IEEE International Conference on Image Processing, San Diego, CA, USA.10.1109/ICIP.2008.4712130 ↗
38. Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. Advances in Neural Information Processing Systems, Curran Associates, Inc.
39. Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Garnett, R. (2016). Improved Techniques for Training GANs. Advances in Neural Information Processing Systems, Curran Associates, Inc.