AB-ResUNet+: Improving Multiple Cardiovascular Structure Segmentation from Computed Tomography Angiography Images
Source abstract
Abstract
Accurate segmentation of cardiovascular structures plays an important role in many clinical applications. Recently, fully convolutional networks (FCNs), led by the UNet architecture, have significantly improved the accuracy and speed of semantic segmentation tasks, greatly improving medical segmentation and analysis tasks. The UNet architecture makes heavy use of contextual information. However, useful channel features are not fully exploited. In this work, we present an improved UNet architecture that exploits residual learning, squeeze and excitation operations, Atrous Spatial Pyramid Pooling (ASPP), and the attention mechanism for accurate and effective segmentation of complex cardiovascular structures and name it AB-ResUNet+. The channel attention block is inserted into the skip connection to optimize the coding ability of each layer. The ASPP block is located at the bottom of the network and acts as a bridge between the encoder and decoder. This increases the field of view of the filters and allows them to include a wider context. The proposed AB-ResUNet+ is evaluated on eleven datasets of different cardiovascular structures, including coronary sinus (CS), descending aorta (DA), inferior vena cava (IVC), left atrial appendage (LAA), left atrial wall (LAW), papillary muscle (PM), posterior mitral leaflet (PML), proximal ascending aorta (PAA), pulmonary aorta (PA), right ventricular wall (RVW), and superior vena cava (SVC). Our experimental evaluations show that the proposed AB-ResUNet+ significantly outperforms the UNet, ResUNet, and ResUNet++ architecture by achieving higher values in terms of Dice coefficient and mIoU.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Medical Image Segmentation Techniques · Advanced Neural Network Applications · Radiomics and Machine Learning in Medical Imaging
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, 31000 Osijek, Croatia
References (47)
Show source-provided references
1. WHO (2021, October 25). Cardiovascular Diseases (CVDs)—Key Facts. Available online: https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds).
2. Overview of the Whole Heart and Heart Chamber Segmentation Methods10.1007/s13239-020-00494-8 ↗
3. Deep Learning for Cardiac Image Segmentation: A Review10.3389/fcvm.2020.00025 ↗
4. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the MICCAI, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
5. Diakogiannis, F.I., Waldner, F., Caccetta, P., and Wu, C. (2019). ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. arXiv.10.1016/j.isprsjprs.2020.01.013 ↗
6. Lin, G., Milan, A., Shen, C., and Reid, I.D. (2016). RefineNet: MultiPath Refinement Networks with Identity Mappings for High-Resolution Semantic Segmentation. arXiv.10.1109/CVPR.2017.549 ↗
7. Huang, G., Liu, Z., and Weinberger, K.Q. (2017, January 21–26). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
8. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation10.1109/TPAMI.2016.2644615 ↗
9. Chen, L.C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv.
10. Jha, D., Smedsrud, P.H., Riegler, M., Johansen, D., de Lange, T., Halvorsen, P., and Johansen, H.D. (2019, January 9–11). ResUNet++: An Advanced Architecture for Medical Image Segmentation. Proceedings of the 2019 IEEE International Symposium on Multimedia (ISM), San Diego, CA, USA.10.1109/ISM46123.2019.00049 ↗
11. Hu, J., Shen, L., Albanie, S., Sun, G., and Wu, E. (2019). Squeeze-and-Excitation Networks. arXiv.10.1109/CVPR.2018.00745 ↗
12. Vaswani, A., Shazeer, N.M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is All you Need. arXiv.
13. Payer, C., Stern, D., Bischof, H., and Urschler, M. (2017, January 10–14). Multi-label Whole Heart Segmentation Using CNNs and Anatomical Label Configurations. Proceedings of the STACOM@MICCAI, Quebec City, QC, Canada.10.1007/978-3-319-75541-0_20 ↗
14. Xu, Z., Wu, Z., and Feng, J. (2018). CFUN: Combining Faster R-CNN and U-net Network for Efficient Whole Heart Segmentation. arXiv.
15. Tong, Q., Ning, M., Si, W., Liao, X., and Qin, J. (2017). 3D Deeply-Supervised U-Net Based Whole Heart Segmentation. International Workshop on Statistical Atlases and Computational Models of the Heart, Springer. Lecture Notes in Computer Science.10.1007/978-3-319-75541-0_24 ↗
16. Yang, X., Bian, C., Yu, L., Ni, D., and Heng, P.A. (2017). 3D Convolutional Networks for Fully Automatic Fine-Grained Whole Heart Partition. International Workshop on Statistical Atlases and Computational Models of the Heart, Springer. Lecture Notes in Computer Science.10.1007/978-3-319-75541-0_19 ↗
17. Yu, L., Cheng, J.Z., Dou, Q., Yang, X., Chen, H., Qin, J., and Heng, P.A. (2017). Automatic 3D Cardiovascular MR Segmentation with Densely-Connected Volumetric ConvNets. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. Lecture Notes in Computer Science.10.1007/978-3-319-66185-8_33 ↗
18. Multi-Depth Fusion Network for Whole-Heart CT Image Segmentation10.1109/ACCESS.2019.2899635 ↗
19. Mortazi, A., Burt, J.R., and Bagci, U. (2017). Multi-Planar Deep Segmentation Networks for Cardiac Substructures from MRI and CT. International Workshop on Statistical Atlases and Computational Models of the Heart, Springer. Lecture Notes in Computer Science.10.1007/978-3-319-75541-0_21 ↗
20. Baumgartner, C.F., Koch, L.M., Pollefeys, M., and Konukoglu, E. (2017). An Exploration of 2D and 3D Deep Learning Techniques for Cardiac MR Image Segmentation. arXiv.10.1007/978-3-319-75541-0_12 ↗
21. Patravali, J., Jain, S., and Chilamkurthy, S. (2017). 2D-3D Fully Convolutional Neural Networks for Cardiac MR Segmentation. arXiv.10.1007/978-3-319-75541-0_14 ↗
22. Jang, Y., Hong, Y., Ha, S., Kim, S., and Chang, H.J. (2017). Automatic Segmentation of LV and RV in Cardiac MRI. International Workshop on Statistical Atlases and Computational Models of the Heart, Springer. Lecture Notes in Computer Science.10.1007/978-3-319-75541-0_17 ↗
23. Luo, C., Shi, C., Li, X., and Gao, D. (2020). Cardiac MR segmentation based on sequence propagation by deep learning. PLoS ONE, 15.10.1371/journal.pone.0230415 ↗
24. Isensee, F., Jaeger, P.F., Full, P.M., Wolf, I., Engelhardt, S., and Maier-Hein, K. (2017). Automatic Cardiac Disease Assessment on cine-MRI via Time-Series Segmentation and Domain Specific Features. arXiv.10.1007/978-3-319-75541-0_13 ↗
25. Deep convolutional neural networks for automatic segmentation of left ventricle cavity from cardiac magnetic resonance images10.1049/iet-cvi.2016.0482 ↗
26. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., and Zhou, Y. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv.
27. Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., and Wang, M. (2021). Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation. arXiv.
28. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. arXiv.10.1109/ICCV48922.2021.00986 ↗
29. Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved?10.1109/TMI.2018.2837502 ↗
30. Challenges and methodologies of fully automatic whole heart segmentation: A review10.1260/2040-2295.4.3.371 ↗
31. Baskaran, L., Al’Aref, S., Maliakal, G., Lee, B., Xu, Z., Choi, J., Lee, S.E., Sung, J., Lin, F., and Dunham, S. (2020). Automatic segmentation of multiple cardiovascular structures from cardiac computed tomography angiography images using deep learning. PLoS ONE, 15.10.1371/journal.pone.0232573 ↗
32. Road Extraction by Deep Residual U-Net10.1109/LGRS.2018.2802944 ↗
33. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
34. Holschneider, M., Kronland-Martinet, R., Morlet, J., and Tchamitchian, P. (1989). A real-time algorithm for signal analysis with the help of the wavelet transform. Wavelets, Springer.10.1007/978-3-642-97177-8_28 ↗
35. Giusti, A., Ciresan, D.C., Masci, J., Gambardella, L.M., and Schmidhuber, J. (2013, January 15–18). Fast image scanning with deep max-pooling convolutional neural networks. Proceedings of the 2013 IEEE International Conference on Image Processing, Melbourne, VIC, Australia.10.1109/ICIP.2013.6738831 ↗
36. Papandreou, G., Kokkinos, I., and Savalle, P.A. (2015, January 7–12). Modeling local and global deformations in Deep Learning: Epitomic convolution, Multiple Instance Learning, and sliding window detection. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298636 ↗
37. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition10.1109/TPAMI.2015.2389824 ↗
38. Zhang, P., Ke, Y., Zhang, Z., Wang, M., Li, P., and Zhang, S. (2018). Urban Land Use and Land Cover Classification Using Novel Deep Learning Models Based on High Spatial Resolution Satellite Imagery. Sensors, 18.10.3390/s18113717 ↗
39. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs10.1109/TPAMI.2017.2699184 ↗
40. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid Scene Parsing Network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
41. Noh, H., Hong, S., and Han, B. (2015, January 11–18). Learning Deconvolution Network for Semantic Segmentation. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.178 ↗
42. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. arXiv.10.1007/978-3-030-01234-2_49 ↗
43. Shahane, S. (2021, October 09). Kaggle Competition: Segmentation of Multiple Cardiovascular Structures. Available online: https://www.kaggle.com/saurabhshahane/segmentation-of-multiple-cardiovascular-structures.
44. Left Atrial Appendage Segmentation Using Fully Convolutional Neural Networks and Modified Three-Dimensional Conditional Random Fields10.1109/JBHI.2018.2794552 ↗
45. Noothout, J.M.H., Vos, B.D., Wolterink, J.M., and Isgum, I. (2018). Automatic segmentation of thoracic aorta segments in low-dose chest CT. arXiv.10.1117/12.2293114 ↗
46. Shi, Z., Zeng, G., Zhang, L., Zhuang, X., Li, L., Yang, G., and Zheng, G. (2018, January 16–20). Bayesian VoxDRN: A Probabilistic Deep Voxelwise Dilated Residual Network for Whole Heart Segmentation from 3D MR Images. Proceedings of the MICCAI 2018, Granada, Spain.10.1007/978-3-030-00937-3_65 ↗
47. dos Santos Tanaka, F.H.K., and de Castro Aranha, C. (2019). Data Augmentation Using GANs. arXiv.