G-UNETR++: A Gradient-Enhanced Network for Accurate and Robust Liver Segmentation from Computed Tomography Images
Source abstract
Abstract
Accurate liver segmentation from computed tomography (CT) scans is essential for liver cancer diagnosis and liver surgery planning. Convolutional neural network (CNN)-based models have limited segmentation performance due to their localized receptive fields. Hybrid models incorporating CNNs and transformers that can capture long-range dependencies have shown promising performance in liver segmentation with the cost of high model complexity. Therefore, a new network architecture named G-UNETR++ is proposed to improve accuracy in liver segmentation with moderate model complexity. Two gradient-based encoders that take the second-order partial derivatives (the first two elements from the last column of the Hessian matrix of a CT scan) as inputs are proposed to learn the 3D geometric features such as the boundaries between different organs and tissues. In addition, a hybrid loss function that combines dice loss, cross-entropy loss, and Hausdorff distance loss is designed to address class imbalance and improve segmentation performance in challenging cases. The proposed method was evaluated on three public datasets, the Liver Tumor Segmentation (LiTS) dataset, the 3D Image Reconstruction for Comparison of Algorithms Database (3D-IRCADb), and the Segmentation of the Liver Competition 2007 (Sliver07) dataset, and achieved 97.38%, 97.50%, and 97.32% in terms of the dice similarity coefficient for liver segmentation on the three datasets, respectively. The proposed method outperformed the other state-of-the-art models on the three datasets, which demonstrated the strong effectiveness, robustness, and generalizability of the proposed method in liver segmentation.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Medical Image Segmentation Techniques · Advanced X-ray and CT Imaging · Radiomics and Machine Learning in Medical Imaging
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Global Entrepreneurship and Information Communication Technology, Handong Global University, Pohang 37554, Republic of Korea
Department of Surgery, Jeonbuk National University Medical School and Hospital, Jeonju 54907, Republic of Korea
Research Institute of Clinical Medicine, Jeonbuk National University, Jeonju 54907, Republic of Korea
Biomedical Research Institute, Jeonbuk National University Hospital, Jeonju 54907, Republic of Korea
Department of Industrial Management and Engineering, Pohang University of Science and Technology, Pohang 37673, Republic of Korea
Funding
Handong Global University Research Grants · Fund of Biomedical Research Institute, Jeonbuk National University Hospital · Mid-Career Researcher Program through the National Research Foundation (NRF) of Korea
References (50)
Show source-provided references
1. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries10.3322/caac.21660 ↗
2. The study and application of the improved region growing algorithm for liver segmentation10.1016/j.ijleo.2013.10.049 ↗
3. A hybrid semi-automatic method for liver segmentation based on level-set methods using multiple seed points10.1016/j.cmpb.2013.08.019 ↗
4. Iterative mesh transformation for 3D segmentation of livers with cancers in CT images10.1016/j.compmedimag.2015.01.006 ↗
5. Dawant, B.M., Li, R., Lennon, B., and Li, S. (2007, January 29). Semi-automatic segmentation of the liver and its evaluation on the MICCAI 2007 grand challenge data set. Proceedings of the MICCAI 2007 Workshop: 3D Segmentation in the Clinic: A Grand Challenge, Brisbane, Australia.
6. Liver segmentation in living liver transplant donors: Comparison of semiautomatic and manual methods10.1148/radiol.2341031801 ↗
7. Jiang, L., Ou, J., Liu, R., Zou, Y., Xie, T., Xiao, H., and Bai, T. (2023). RMAU-Net: Residual Multi-Scale Attention U-Net for liver and tumor segmentation in CT images. Comput. Biol. Med., 158.10.1016/j.compbiomed.2023.106838 ↗
8. Chen, Y., Zheng, C., Zhou, T., Feng, L., Liu, L., Zeng, Q., and Wang, G. (2023). A deep residual attention-based U-Net with a biplane joint method for liver segmentation from CT scans. Comput. Biol. Med., 152.10.1016/j.compbiomed.2022.106421 ↗
9. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention MICCAI International Conference, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
10. Gul, S., Khan, M.S., Bibi, A., Khandakar, A., Ayari, M.A., and Chowdhury, M.E.H. (2022). Deep learning techniques for liver and liver tumor segmentation: A review. Comput. Biol. Med., 147.10.1016/j.compbiomed.2022.105620 ↗
11. UNesT: Local spatial representation learning with hierarchical transformer for efficient medical segmentation10.1016/j.media.2023.102939 ↗
12. Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H.R., and Xu, D. (2022, January 3–8). UNETR: Transformers for 3d medical image segmentation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACV51458.2022.00181 ↗
13. Hu, H., Zhang, Z., Xie, Z., and Lin, S. (November, January 27). Local relation networks for image recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.
14. Mohajelin, F., Sheykhivand, S., Shabani, A., Danishvar, M., Danishvar, S., and Lahijan, L.Z. (2024). Automatic recognition of multiple emotional classes from EEG signals through the use of graph theory and convolutional neural networks. Sensors, 24.10.3390/s24185883 ↗
15. Ardabili, S.Z., Bahmani, S., Lahijan, L.Z., Khaleghi, N., Sheykhivand, S., and Danishvar, S. (2024). A novel approach for automatic detection of driver fatigue using EEG signals based on graph convolutional networks. Sensors, 24.10.3390/s24020364 ↗
16. Khoshkhabar, M., Meshgini, S., Afrouzian, R., and Danishvar, S. (2023). Automatic liver tumor segmentation from CT images using graph convolutional network. Sensors, 23.10.3390/s23177561 ↗
17. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., and Zhou, Y. (2021). Transunet: Transformers make strong encoders for medical image segmentation. arXiv.
18. Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H., and Xu, D. (2022). Swin UNETR: Swin transformers for semantic segmentation of brain tumors in MRI images. arXiv.10.1007/978-3-031-08999-2_22 ↗
19. Zhou, H.-Y., Guo, J., Zhang, Y., Yu, L., Wang, L., and Yu, Y. (2021). nnFormer: Interleaved transformer for volumetric segmentation. arXiv.
20. UNETR++: Delving into efficient and accurate 3D medical image segmentation10.1109/TMI.2024.3398728 ↗
21. Landman, B., Xu, Z., Igelsias, J.E., Styner, M., Langerak, T.R., and Klein, A. (2015, January 7–12). 2015 MICCAI multi-atlas labeling beyond the cranial vault workshop and challenge. Proceedings of the MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge, Boston, MA, USA.
22. Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., and Ronneberger, O. (2016, January 17–21). 3D U-Net: Learning dense volumetric segmentation from sparse annotation. Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention, Athens, Greece.10.1007/978-3-319-46723-8_49 ↗
23. Dou, Q., Chen, H., Jin, Y., Yu, L., Qin, J., and Heng, P.-A. (2016, January 17–21). 3D deeply supervised network for automatic liver segmentation from CT volumes. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Athens, Greece.10.1007/978-3-319-46723-8_18 ↗
24. Automatic multi-organ segmentation on abdominal CT with dense V-networks10.1109/TMI.2018.2806309 ↗
25. Milletari, F., Navab, N., and Ahmadi, S.-A. (2016, January 25–28). V-Net: Fully convolutional neural networks for volumetric medical image segmentation. Proceedings of the 2016 4th International Conference on 3D Vision (3DV), Stanford, CA, USA.10.1109/3DV.2016.79 ↗
26. Roth, H.R., Oda, H., Hayashi, Y., Oda, M., Shimizu, N., Fujiwara, M., Misawa, K., and Mori, K. (2017). Hierarchical 3D fully convolutional networks for multi-organ segmentation. arXiv.10.1007/978-3-030-00937-3_48 ↗
27. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
28. Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., and Wang, M. (2021). Swin-Unet: U-Net-like pure transformer for medical image segmentation. arXiv.
29. Karimi, D., Vasylechko, S.D., and Gholipour, A. (2021). Convolution-free medical image segmentation using transformers. arXiv.10.1007/978-3-030-87193-2_8 ↗
30. Khan, A., Rauf, Z., Sohail, A., Rehman, A., Asif, H.M., Asif, A., and Farooq, U. (2023). A survey of the vision transformers and its CNN-transformer based variants. arXiv.10.1007/s10462-023-10595-0 ↗
31. Valanarasu, J.M.J., Oza, P., Hacihaliloglu, I., and Patel, V.M. (October, January 27). Medical transformer: Gated axial-attention for medical image segmentation. Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention, Strasbourg, France.
32. Xie, Y., Zhang, J., Shen, C., and Xia, Y. (October, January 27). CoTr: Efficiently bridging CNN and transformer for 3D medical image segmentation. Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention, Strasbourg, France.
33. Wang, W., Chen, C., Ding, M., Yu, H., Zha, S., and Li, J. (October, January 27). TransBTS: Multimodal brain tumor segmentation using transformer. Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention, Strasbourg, France.
34. MCFA-UNet: Multiscale cascaded feature attention U-Net for liver segmentation10.1016/j.irbm.2023.100789 ↗
35. Automatic liver segmentation by integrating fully convolutional networks into active contour models10.1002/mp.13735 ↗
36. Bridging the gap between 2D and 3D contexts in CT volume for liver and tumor segmentation10.1109/JBHI.2021.3075752 ↗
37. DefED-Net: Deformable encoder-decoder network for liver and liver tumor segmentation10.1109/TRPMS.2021.3059780 ↗
38. Hybrid-attention densely connected U-Net with GAP for extracting livers from CT volumes10.1002/mp.15435 ↗
39. Kushnure, D.T., and Talbar, S.N. (2022). HFRU-Net: High-level feature fusion and recalibration UNet for automatic liver and tumor segmentation in CT images. Comput. Methods Programs Biomed., 213.10.1016/j.cmpb.2021.106501 ↗
40. DHT-Net: Dynamic hierarchical transformer network for liver and tumor Segmentation10.1109/JBHI.2023.3268218 ↗
41. Zhu, J., Liu, Z., Gao, W., and Fu, Y. (2024). CotepRes-Net: An efficient U-Net based deep learning method of liver segmentation from Computed Tomography images. Biomed. Signal Process. Control, 88.10.1016/j.bspc.2023.105660 ↗
42. Reducing the Hausdorff distance in medical image segmentation with convolutional neural networks10.1109/TMI.2019.2930068 ↗
43. Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., and Liang, J. (2018). Unet++: A nested u-net architecture for medical image segmentation. arXiv.10.1007/978-3-030-00889-5_1 ↗
44. Lee, C.-Y., Xie, S., Gallagher, P., Zhang, Z., and Tu, Z. (2014). Deeply-supervised nets. arXiv.
45. Kingma, D.P., and Ba, J. (2017). Adam: A method for stochastic optimization. arXiv.
46. Chibuike, O., and Yang, X. (2024). Convolutional Neural Network–Vision Transformer Architecture with Gated Control Mechanism and Multi-Scale Fusion for Enhanced Pulmonary Disease Classification. Diagnostics, 14.10.3390/diagnostics14242790 ↗
47. MA-Net: A multi-scale attention network for liver and tumor segmentation10.1109/ACCESS.2020.3025372 ↗
48. Sakboonyara, B., and Taeprasartsit, P. (2019, January 23–26). U-Net and mean-shift histogram for efficient liver segmentation from CT images. Proceedings of the 11th International Conference on Knowledge and Smart Technology (KST), Phuket, Thailand.10.1109/KST.2019.8687816 ↗
49. Modified U-Net (mU-Net) with incorporation of object-dependent high level features for improved liver and liver-tumor segmentation in CT images10.1109/TMI.2019.2948320 ↗
50. Cascaded deep convolutional encoder-decoder neural networks for efficient liver tumor segmentation10.1016/j.mehy.2019.109431 ↗