journal article·2025-07-11Open access

Generative Adversarial Networks in Histological Image Segmentation: A Systematic Literature Review

Yanna Leidy Ketley Fernandes Cruz, Antonio Fhillipi Maciel Silva, Ewaldo Eder Carvalho Santana, Daniel G. Costa

Applied Sciences· Volume 15· Issue 14

Abstract

Histological image analysis plays a crucial role in understanding and diagnosing various diseases, but manually segmenting these images is often complex, time-consuming, and heavily reliant on expert knowledge. Generative adversarial networks (GANs) have emerged as promising tools to assist in this task, enhancing the accuracy and efficiency of segmentation in histological images. This systematic literature review aims to explore how GANs have been utilized for segmentation in this field, highlighting the latest trends, key challenges, and opportunities for future research. The review was conducted across multiple digital libraries, including IEEE, Springer, Scopus, MDPI, and PubMed, with combinations of the keywords “generative adversarial network” or “GAN”, “segmentation” or “image segmentation” or “semantic segmentation”, and “histology” or “histological” or “histopathology” or “histopathological”. We reviewed 41 GAN-based histological image segmentation articles published between December 2014 and February 2025. We summarized and analyzed these papers based on the segmentation regions, datasets, GAN tasks, segmentation tasks, and commonly used metrics. Additionally, we discussed advantages, challenges, and future research directions. The analyzed studies demonstrated the versatility of GANs in handling challenges like stain variability, multi-task segmentation, and data scarcity—all crucial challenges in the analysis of histopathological images. Nevertheless, the field still faces important challenges, such as the need for standardized datasets, robust evaluation metrics, and better generalization across diverse tissues and conditions.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: AI in cancer detection · Generative Adversarial Networks and Image Synthesis · Cell Image Analysis Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Graduate Program in Electrical Engineering, Federal University of Maranhão (UFMA), São Luís 65080-805, Brazil

Computer Science Department, State University of Piauí (UESPI), Floriano 64800-000, Brazil

Graduate Program in Computer and Systems Engineering, State University of Maranhão (UEMA), São Luís 65081-400, Brazil

SYSTEC-ARISE, Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal

Funding

Coordination for the Improvement of Higher Education Personnel (CAPES) · Foundation for Research Support and Scientific and Technological Development of Maranhão (FAPEMA) · Foundation for Research Support of Piauí (FAPEPI) · Foundation of the State University of Piauí (UESPI)

References (100+)

Show source-provided references
1. Mezei, T., Kolcsár, M., Joó, A., and Gurzu, S. (2024). Image Analysis in Histopathology and Cytopathology: From Early Days to Current Perspectives. J. Imaging, 10.10.3390/jimaging10100252 ↗
2. Ali, M., Benfante, V., Basirinia, G., Alongi, P., Sperandeo, A., Quattrocchi, A., Giannone, A.G., Cabibi, D., Yezzi, A., and Raimondo, D.D. (2025). Applications of Artificial Intelligence, Deep Learning, and Machine Learning to Support the Analysis of Microscopic Images of Cells and Tissues. J. Imaging, 11.10.3390/jimaging11020059 ↗
3. The Wonderful Colors of the Hematoxylin–Eosin Stain in Diagnostic Surgical Pathology10.1177/1066896913517939 ↗
4. Stain normalization methods for histopathology image analysis: A comprehensive review and experimental comparison10.1016/j.inffus.2023.101997 ↗
5. Chen, Y., Yang, X.H., Wei, Z., Heidari, A.A., Zheng, N., Li, Z., Chen, H., Hu, H., Zhou, Q., and Guan, Q. (2022). Generative Adversarial Networks in Medical Image augmentation: A review. Comput. Biol. Med., 144.10.1016/j.compbiomed.2022.105382 ↗
6. Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., and Saltz, J.H. (2019, January 15–20). Robust Histopathology Image Analysis: To Label or to Synthesize?. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00873 ↗
7. Vo, T., and Khan, N. (2022, January 11–15). Edge-preserving image synthesis for unsupervised domain adaptation in medical image segmentation. Proceedings of the 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Glasgow, UK.10.1109/EMBC48229.2022.9871402 ↗
8. Weakly supervised deep nuclei segmentation using points annotation in histopathology images
9. Saad, M., Hossain, M.A., and Tizhoosh, H.R. (2022). A Survey on Training Challenges in Generative Adversarial Networks for Biomedical Image Analysis. arXiv.
10. Lee, M. (2023). Recent Advances in Generative Adversarial Networks for Gene Expression Data: A Comprehensive Review. Mathematics, 11.10.3390/math11143055 ↗
11. Liu, L., Xia, Y., and Tang, L. (2020, January 11–13). An overview of biological data generation using generative adversarial networks. Proceedings of the 2020 IEEE Conference on Telecommunications, Optics and Computer Science (TOCS), Shenyang, China.10.1109/TOCS50858.2020.9339748 ↗
12. Generative adversarial networks and its applications in the biomedical image segmentation: A comprehensive survey10.1007/s13735-022-00240-x ↗
13. Hörst, F., Rempe, M., Heine, L., Seibold, C., Keyl, J., Baldini, G., Ugurel, S., Siveke, J., Grünwald, B., and Egger, J. (2023). CellViT: Vision Transformers for Precise Cell Segmentation and Classification. arXiv.10.1016/j.media.2024.103143 ↗
14. Obeid, A., Boumaraf, S., Sohail, A., Hassan, T., Javed, S., Dias, J., Bennamoun, M., and Werghi, N. (2024). Advancing Histopathology with Deep Learning Under Data Scarcity: A Decade in Review. arXiv.
15. Lahreche, F., Moussaoui, A., and Oulad-Naoui, S. (2024, January 26–27). Medical Image Semantic Segmentation Using Deep Learning: A Survey. Proceedings of the International Conference on Emerging Intelligent Systems for Sustainable Development (ICEIS 2024), Aflou, Algeria.10.2991/978-94-6463-496-9_25 ↗
16. Sultan, B., Rehman, A., and Riyaz, L. (2024). Generative Adversarial Networks in the Field of Medical Image Segmentation. Deep Learning Applications in Medical Image Segmentation: Overview, Approaches, and Challenges, Wiley.10.1002/9781394245369.ch8 ↗
17. A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications10.1109/TKDE.2021.3130191 ↗
18. Alhumaid, M., Alhumaid, M., and Fayoumi, A. (2024). Transfer Learning-Based Classification of Maxillary Sinus Using Generative Adversarial Networks. Appl. Sci., 14.10.3390/app14073083 ↗
19. Generative Adversarial Networks (GANs) in Medical Imaging: Advancements, Applications, and Challenges10.1109/ACCESS.2024.3370848 ↗
20. Generative Adversarial Networks in Digital Pathology and Histopathological Image Processing: A Review10.4103/jpi.jpi_103_20 ↗
21. Zhao, J., Hou, X., Pan, M., and Zhang, H. (2022). Attention-based generative adversarial network in medical imaging: A narrative review. Comput. Biol. Med., 149.10.1016/j.compbiomed.2022.105948 ↗
22. Hussain, J., Båth, M., and Ivarsson, J. (2025). Generative adversarial networks in medical image reconstruction: A systematic literature review. Comput. Biol. Med., 191.10.1016/j.compbiomed.2025.110094 ↗
23. Xun, S., Li, D., Zhu, H., Chen, M., Wang, J., Li, J., Chen, M., Wu, B., Zhang, H., and Chai, X. (2022). Generative adversarial networks in medical image segmentation: A review. Comput. Biol. Med., 140.10.1016/j.compbiomed.2021.105063 ↗
24. Literature searches in systematic reviews and meta-analyses: A review, evaluation, and recommendations10.1016/j.jvb.2020.103377 ↗
25. Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Shamseer, L., Tetzlaff, J.M., Akl, E.A., and Brennan, S.E. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372.10.1136/bmj.n71 ↗
26. Ibrahim, M., Khalil, Y.A., Amirrajab, S., Sun, C., Breeuwer, M., Pluim, J., Elen, B., Ertaylan, G., and Dumontier, M. (2025). Generative AI for synthetic data across multiple medical modalities: A systematic review of recent developments and challenges. Comput. Biol. Med., 189.10.1016/j.compbiomed.2025.109834 ↗
27. Haggerty, J., Wang, X., Dickinson, A., O’Malley, C., and Martin, E. (2014). Segmentation of epidermal tissue with histopathological damage in images of haematoxylin and eosin stained human skin. BMC Med. Imaging, 14.10.1186/1471-2342-14-7 ↗
28. Mahbod, A., Schaefer, G., Dorffner, G., Hatamikia, S., Ecker, R., and Ellinger, I. (2022). A dual decoder U-Net-based model for nuclei instance segmentation in hematoxylin and eosin-stained histological images. Front. Med., 9.10.3389/fmed.2022.978146 ↗
29. Histology image analysis for carcinoma detection and grading10.1016/j.cmpb.2011.12.007 ↗
30. Segmentation of histological images and fibrosis identification with a convolutional neural network10.1016/j.compbiomed.2018.05.015 ↗
31. Deep learning models for automatic tumor segmentation and total tumor volume assessment in patients with colorectal liver metastases10.1186/s41747-023-00383-4 ↗
32. Schoenpflug, L.A., Lafarge, M.W., Frei, A.L., and Koelzer, V.H. (2023). Multi-task learning for tissue segmentation and tumor detection in colorectal cancer histology slides. arXiv.
33. Gudhe, N.R., Kosma, V.M., Behravan, H., and Mannermaa, A. (2023). Nuclei instance segmentation from histopathology images using Bayesian dropout based deep learning. BMC Med. Imaging, 23.10.1186/s12880-023-01121-3 ↗
34. A novel deep learning-based algorithm combining histopathological features with tissue areas to predict colorectal cancer survival from whole-slide images10.1186/s12967-023-04530-8 ↗
35. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative Adversarial Networks. Adv. Neural Inf. Process. Syst., 27.
36. Survey: Application and analysis of generative adversarial networks in medical images10.1007/s10462-024-10992-z ↗
37. Kodali, N., Abernethy, J., Hays, J., and Kira, Z. (2017). On Convergence and Stability of GANs. arXiv.
38. Fedus, W., Goodfellow, I., and Dai, A.M. (2023). Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence at Every Step. arXiv.
39. Lucic, M., Kurach, K., Michalski, M., Gelly, S., and Bousquet, O. (2018). Are GANs Created Equal? A Large-Scale Study. Adv. Neural Inf. Process. Syst. NeurIPS, 31.
40. Zhang, H., Yu, Y., Jojic, N., Xiao, H., Wang, Y., Liang, Y., and Hsieh, C.J. (2020, January 13–18). Overfitting in Adversarially Robust Deep Learning. Proceedings of the 37th International Conference on Machine Learning (ICML), Virtual.
41. Mirza, M., and Osindero, S. (2014). Conditional generative adversarial nets. arXiv.
42. Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
43. Karras, T., Laine, S., and Aila, T. (2019, January 15–20). A style-based generator architecture for generative adversarial networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00453 ↗
44. Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., and Aila, T. (2020, January 14–19). Analyzing and improving the image quality of StyleGAN. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00813 ↗
45. Karras, T., Aila, T., Laine, S., and Lehtinen, J. (May, January 30). Progressive growing of GANs for improved quality, stability, and variation. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
46. Han, C.C., Rundo, L., Araki, R., Tang, Y.C., Peng, S.L., Nakayama, H., Lin, C.C., and Li, C.H. (2019). Infinite Brain MR Images: PGGAN-based Data Augmentation for Tumor Detection. arXiv.10.1007/978-981-13-8950-4_27 ↗
47. Korkinof, D., Rijken, T., O’Neill, M., Matthew, J., and Glocker, B. (2018). High-Resolution Mammogram Synthesis using Progressive Generative Adversarial Networks. arXiv.
48. Xue, Y., Ye, J., Zhou, Q., You, H., Ouyang, W., and Do, Q.V. (2021). Selective Synthetic Augmentation with HistoGAN for Improved Histopathology Image Classification. arXiv.10.1016/j.media.2020.101816 ↗
49. Arjovsky, M., Chintala, S., and Bottou, L. (2017). Wasserstein GAN. arXiv.
50. GANs for medical image analysis10.1016/j.artmed.2020.101938 ↗
51. Generative adversarial network in medical imaging: A review10.1016/j.media.2019.101552 ↗
52. Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21–26). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.632 ↗
53. The Role of Protocol Papers, Scoping Reviews, and Systematic Reviews in Responsible AI Research10.1109/MTS.2025.3558666 ↗
54. PROSPERO (2025, February 10). International Prospective Register of Systematic Reviews. Centre for Reviews and Dissemination, University of York., Available online: https://www.crd.york.ac.uk/prospero/.
55. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification10.1016/j.neucom.2018.09.013 ↗
56. Unsupervised multi-domain progressive stain transfer guided by style encoding dictionary10.1109/TIP.2024.3349866 ↗
57. Liu, Y., Wagner, S.J., and Peng, T. (2022). Multi-modality microscopy image style augmentation for nuclei segmentation. J. Imaging, 8.10.3390/jimaging8030071 ↗
58. Cross-boosted multi-target domain adaptation for multi-modality histopathology image translation and segmentation10.1109/JBHI.2022.3153793 ↗
59. Unsupervised learning for cell-level visual representation in histopathology images with generative adversarial networks10.1109/JBHI.2018.2852639 ↗
60. Stain-adaptive self-supervised learning for histopathology image analysis10.1016/j.patcog.2024.111242 ↗
61. Image-to-image translation of label-free molecular vibrational images for a histopathological review using the UNet+/seg-cGAN model10.1364/BOE.445319 ↗
62. SynCLay: Interactive synthesis of histology images from bespoke cellular layouts10.1016/j.media.2023.102995 ↗
63. A laplacian pyramid based generative h&e stain augmentation network10.1109/TMI.2023.3317239 ↗
64. MASG-GAN: A multi-view attention superpixel-guided generative adversarial network for efficient and simultaneous histopathology image segmentation and classification10.1016/j.neucom.2021.08.039 ↗
65. Learning to generalize over subpartitions for heterogeneity-aware domain adaptive nuclei segmentation10.1007/s11263-024-02004-y ↗
66. SAFRON: Stitching across the frontier network for generating colorectal cancer histology images10.1016/j.media.2021.102337 ↗
67. Nucleus-aware self-supervised pretraining using unpaired image-to-image translation for histopathology images10.1109/TMI.2023.3309971 ↗
68. GenSelfDiff-HIS: Generative Self-Supervision Using Diffusion for Histopathological Image Segmentation10.1109/TMI.2024.3453492 ↗
69. Tackling stain variability using CycleGAN-based stain augmentation10.1016/j.jpi.2022.100140 ↗
70. Du, Z., Zhang, P., Huang, X., Hu, Z., Yang, G., Xi, M., and Liu, D. (2025). Deeply supervised two stage generative adversarial network for stain normalization. Sci. Rep., 15.10.1038/s41598-025-91587-8 ↗
71. MiNuGAN: Dual segmentation of mitoses and nuclei using conditional GANs on multi-center breast H&E images10.1016/j.jpi.2022.100002 ↗
72. HistoStarGAN: A unified approach to stain normalisation, stain transfer and stain invariant segmentation in renal histopathology10.1016/j.knosys.2023.110780 ↗
73. Application of histopathology image analysis using deep learning networks10.1007/s44230-024-00077-z ↗
74. Deep learning-based virtual staining, segmentation, and classification in label-free photoacoustic histology of human specimens10.1038/s41377-024-01554-7 ↗
75. Adversarial stain transfer for histopathology image analysis10.1109/TMI.2017.2781228 ↗
76. Regional realness-aware generative adversarial networks for stain normalization10.1007/s00521-023-08659-x ↗
77. Lafarge, M.W., Pluim, J.P., Eppenhof, K.A., and Veta, M. (2019). Learning domain-invariant representations of histological images. Front. Med., 6.10.3389/fmed.2019.00162 ↗
78. Shahini, A., Gambella, A., Molinari, F., and Salvi, M. (2025). Semantic-driven synthesis of histological images with controllable cellular distributions. Comput. Methods Programs Biomed., 261.10.1016/j.cmpb.2025.108621 ↗
79. Which pixel to annotate: A label-efficient nuclei segmentation framework10.1109/TMI.2022.3221666 ↗
80. Wang, H., Xu, G., Pan, X., Liu, Z., Lan, R., and Luo, X. (2022). Multi-task generative adversarial learning for nuclei segmentation with dual attention and recurrent convolution. Biomed. Signal Process. Control, 75.10.1016/j.bspc.2022.103558 ↗
81. Super-resolution and segmentation deep learning for breast cancer histopathology image analysis10.1364/BOE.463839 ↗
82. Wang, L., Zhang, S., Gu, L., Zhang, J., Zhai, X., Sha, X., and Chang, S. (2021). Automatic consecutive context perceived transformer GAN for serial sectioning image blind inpainting. Comput. Biol. Med., 136.10.1016/j.compbiomed.2021.104751 ↗
83. Domain adaptation-based deep learning for automated tumor cell (TC) scoring and survival analysis on PD-L1 stained tissue images10.1109/TMI.2021.3081396 ↗
84. Azam, A.B., Wee, F., Väyrynen, J.P., Yim, W.W.Y., Xue, Y.Z., Chua, B.L., Lim, J.C.T., Somasundaram, A.C., Tan, D.S.W., and Takano, A. (2024). Training immunophenotyping deep learning models with the same-section ground truth cell label derivation method improves virtual staining accuracy. Front. Immunol., 15.10.3389/fimmu.2024.1404640 ↗
85. Seamless virtual whole slide image synthesis and validation using perceptual embedding consistency10.1109/JBHI.2020.2975151 ↗
86. Kweon, J., Yoo, J., Kim, S., Won, J., and Kwon, S. (2022). A novel method based on GAN using a segmentation module for oligodendroglioma pathological image generation. Sensors, 22.10.3390/s22103960 ↗
87. Dataset of segmented nuclei in hematoxylin and eosin stained histopathology images of ten cancer types10.1038/s41597-020-0528-1 ↗
88. Deep adversarial training for multi-organ nuclei segmentation in histopathology images10.1109/TMI.2019.2927182 ↗
89. Enhanced pathology image quality with restore–generative adversarial network10.1016/j.ajpath.2022.12.011 ↗
90. Truncated normal mixture prior based deep latent model for color normalization of histology images10.1109/TMI.2023.3238425 ↗
91. The segmentation of nuclei from histopathology images with synthetic data10.1007/s11760-023-02597-w ↗
92. Conditional GANs based system for fibrosis detection and quantification in Hematoxylin and Eosin whole slide images10.1016/j.media.2022.102537 ↗
93. Residual cyclegan for robust domain transformation of histopathological tissue slides10.1016/j.media.2021.102004 ↗
94. Towards histopathological stain invariance by unsupervised domain augmentation using generative adversarial networks10.1016/j.neucom.2021.07.005 ↗
95. Deepfake histologic images for enhancing digital pathology10.1016/j.labinv.2022.100006 ↗
96. Ruiz-Casado, J.L., Molina-Cabello, M.A., and Luque-Baena, R.M. (2024). Enhancing Histopathological Image Classification Performance through Synthetic Data Generation with Generative Adversarial Networks. Sensors, 24.10.20944/preprints202405.0467.v1 ↗
97. Generative adversarial networks for facilitating stain-independent supervised and unsupervised segmentation: A study on kidney histology10.1109/TMI.2019.2899364 ↗
98. Bai, B., Yang, X., Li, Y., Zhang, Y., Pillar, N., and Ozcan, A. (2022). Deep Learning-enabled Virtual Histological Staining of Biological Samples. arXiv.10.1038/s41377-023-01104-7 ↗
99. Xu, Z., Huang, X., Fernández Moro, C., Bozóky, B., and Zhang, Q. (2019). GAN-based Virtual Re-Staining: A Promising Solution for Whole Slide Image Analysis. arXiv.
100. Smith, J., and Doe, J. (2018, January 4–7). High-Resolution Generative Adversarial Neural Networks Applied to Histological Images Generation. Proceedings of the ICANN 2018, Rhodes, Greece.