Multi-Task Learning for Small Brain Tumor Segmentation from MRI
Source abstract
Abstract
Segmenting brain tumors accurately and reliably is an essential part of cancer diagnosis and treatment planning. Brain tumor segmentation of glioma patients is a challenging task because of the wide variety of tumor sizes, shapes, positions, scanning modalities, and scanner’s acquisition protocols. Many convolutional neural network (CNN) based methods have been proposed to solve the problem of brain tumor segmentation and achieved great success. However, most previous studies do not fully take into account multiscale tumors and often fail to segment small tumors, which may have a significant impact on finding early-stage cancers. This paper deals with the brain tumor segmentation of any sizes, but specially focuses on accurately identifying small tumors, thereby increasing the performance of the brain tumor segmentation of overall sizes. Instead of using heavyweight networks with multi-resolution or multiple kernel sizes, we propose a novel approach for better segmentation of small tumors by dilated convolution and multi-task learning. Dilated convolution is used for multiscale feature extraction, however it does not work well with very small tumor segmentation. For dealing with small-sized tumors, we try multi-task learning, where an auxiliary task of feature reconstruction is used to retain the features of small tumors. The experiment shows the effectiveness of segmenting small tumors with the proposed method. This paper contributes to the detection and segmentation of small tumors, which have seldom been considered before and the new development of hierarchical analysis using multi-task learning.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Brain Tumor Detection and Classification · Advanced Neural Network Applications · Medical Image Segmentation Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Dept. of Electronics and Computer Engineering, Chonnam National University, 77 Yongbong-ro, Gwangju 500-757, Korea
Funding
National Research Foundation of Korea
References (60)
Show source-provided references
1. A Survey of MRI- Based Medical Image Analysis for Brain Tumor Studies10.1088/0031-9155/58/13/R97 ↗
2. Health-Related Quality of Life in High-Grade Glioma Patients10.5732/cjc.013.10214 ↗
3. Detection and Localization of Early-Stage Multiple Brain Tumors Using Ahybrid Technique of Patch-Based Processing, K-Means Clustering and Object Counting10.1155/2020/9035096 ↗
4. Pries, T.P., Jahan, R., and Suman, P. (2018, January 14–15). Review of Brain Tumor Segmentation, Detection and Classification Algorithms in fMRI Images. Proceedings of the 2018 International Conference on Computational and Characterization Techniques in Engineering & Sciences (CCTES), Lucknow, India.10.1109/CCTES.2018.8674150 ↗
5. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
6. Hamaguchi, R., Fujita, A., Nemoto, K., Imaizumi, T., and Hikosaka, S. (2018, January 12–15). Effective Use of Dilated Convolutions for Segmenting Small Object Instances in Remote Sensing Imagery. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.10.1109/WACV.2018.00162 ↗
7. Li, Y., Qi, H., Dai, J., Ji, X., and Wei, Y. (2017, January 21–26). Fully Convolutional Instance-Aware Semantic Segmentation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.472 ↗
8. Shareef, B., Xian, M., and Vakanski, A. (2020, January 3–7). Stan: Small Tumor-Aware Network for Breast Ultrasound Image Segmentation. Proceedings of the 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), Iowa City, IA, USA.10.1109/ISBI45749.2020.9098691 ↗
9. Zhang, K., Zhou, X., and Wu, J. (2019, January 22–25). U-Module: Better Parameters Initialization of Convolutional Neural Network for Medical Image Classification. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8803799 ↗
10. Kuang, Z., Li, Z., Zhao, T., and Fan, J. (2017, January 19–21). Deep Multi-Task Learning for Large-Scale Image Classification. Proceedings of the 2017 IEEE Third International Conference on Multimedia Big Data (BigMM), Laguna Hills, CA, USA.10.1109/BigMM.2017.72 ↗
11. Li, L., and Gong, B. (2019, January 7–11). End-to-End Video Captioning with Multitask Reinforcement Learning. Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa Village, HI, USA.10.1109/WACV.2019.00042 ↗
12. Geon, W.L., and Hong, K.K. (2020). Multi-Task Learning U-Net for Single-Channel Speech Enhancement and Mask-Based Voice Activity Detection. Appl. Sci., 10.10.3390/app10093230 ↗
13. Multi-Task Learning for the Segmentation of Organs at Risk with Label Dependence10.1016/j.media.2020.101666 ↗
14. Abdullah, A.Z.I., and Demetri, T. (2019, January 13). Semi-supervised Multi-task Learning with Chest X-Ray Images. Proceedings of the International Workshop on Machine Learning in Medical Imaging, Shenzhen, China.
15. Deep Learning in Medical Image Analysis10.1146/annurev-bioeng-071516-044442 ↗
16. Computer Aided Diagnosis System for Breast Cancer Based on Color Doppler Flow Imaging10.1007/s10916-012-9869-4 ↗
17. A Perspective on Deep Imaging10.1109/ACCESS.2016.2624938 ↗
18. Machine Learning Methods for Binary and Multiclass Classification of Melanoma Thickness from Dermoscopic Images10.1109/TMI.2015.2506270 ↗
19. Incremental Learning with Selective Memory (ILSM): Towards Fast Prostate Localization for Image Guided Radiotherapy
20. Robust Learning-Based Parsing and Annotation of Medical Radiographs10.1109/TMI.2010.2077740 ↗
21. Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions10.1007/s10278-017-9983-4 ↗
22. Khan, G., and Khan, N.M. (2018, January 3–6). Brain MRI Segmentation using efficient 3D Fully Convolutional Neural Networks. Proceedings of the 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Madrid, Spain.10.1109/BIBM.2018.8621509 ↗
23. Fully Automatic Liver Segmentation Combining Multi-Dimensional Graph Cut with Shape Information in 3D CT Images
24. Valindria, V.V., Lavdas, I., Cerrolaza, J., Aboagye, E.O., Rockall, A.G., Rueckert, D., and Glocker, B. (2018, January 16). Small Organ Segmentation in Whole-Body MRI Using a Two-Stage FCN and Weighting Schemes. Proceedings of the International Workshop on Machine Learning in Medical Imaging, Granada, Spain.10.1007/978-3-030-00919-9_40 ↗
25. Automatic Lung Segmentation for Accurate Quantitation of Volumetric X-Ray CT Images10.1109/42.929615 ↗
26. Automatic Cardiac Ventricle Segmentation in MR Images: A Validation Study10.1007/s11548-010-0532-6 ↗
27. An Unsupervised Learning Method with a Clustering Approach for Tumor Identification and Tissue Segmentation in Magnetic Resonance Brain Images10.1016/j.asoc.2015.09.016 ↗
28. A Multi-Scale 3D Otsu Thresholding Algorithm for Medical Image Segmentation10.1016/j.dsp.2016.08.003 ↗
29. A Hybrid Edge-Based Segmentation Approach for Ultrasound Medical Images10.1016/j.bspc.2016.06.012 ↗
30. On Image Segmentation Methods Applied to Glioblastoma: State of Art and New Trends10.1016/j.irbm.2015.12.004 ↗
31. A Brain Tumor Segmentation Framework Based on Outlier Detection10.1016/j.media.2004.06.007 ↗
32. GLISTR: Glioma Image Segmentation and Registration10.1109/TMI.2012.2210558 ↗
33. Kwon, D., Shinohara, R.T., Akbari, H., and Davatzikos, C. (2014, January 14–18). Combining Generative Models for Multifocal Glioma Segmentation and Registration. Proceedings of the Multimodal Brain Image Analysis, Boston, MA, USA.10.1007/978-3-319-10404-1_95 ↗
34. A Patch-Based Approach for the Segmentation of Pathologies: Application to Glioma Labelling10.1109/TMI.2015.2508150 ↗
35. Cobzas, D., Birkbeck, N., Schmidt, M., Jagersand, M., and Murtha, A. (2007, January 14–20). 3D Variational Brain Tumor Segmentation using a High Dimensional Feature Set. Proceedings of the 2007 IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Brazil.10.1109/ICCV.2007.4409130 ↗
36. 3D Variational Brain Tumor Segmentation on a Clustered Feature Set
37. Optimal Symmetric Multimodal Templates and Concatenated Random Forests for Supervised Brain Tumor Segmentation (Simplified) with ANTsR10.1007/s12021-014-9245-2 ↗
38. Le Folgoc, L., Nori, A.V., Ancha, S., and Criminisi, A. (2016, January 17–21). Lifted Auto-Context Forests for Brain Tumour Segmentation. Proceedings of the International Workshop on Brain Lesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Athens, Greece.10.1007/978-3-319-55524-9_17 ↗
39. Bauer, S., Nolte, L.-P., and Reyes, M. (2011, January 18–22). Fully Automatic Segmentation of Brain Tumor Images Using Support Vector Machine Classification in Combination with Hierarchical Conditional Random Field Regularization. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Toronto, ON, Canada.10.1007/978-3-642-23626-6_44 ↗
40. Lee, C.-H., Schmidt, M., Murtha, A., Bistritz, A., Sander, J., and Greiner, R. (2005, January 21). Segmenting Brain Tumors with Conditional Random Fields and Support Vector Machines. Proceedings of the International Workshop on Computer Vision for Biomedical Image Applications, Beijing, China.10.1007/11569541_47 ↗
41. Geremia, E., Menze, B.H., and Ayache, N. (2012, January 1). Spatial Decision Forests for Glioma Segmentation in Multi-Channel Mr Images. Proceedings of the MICCAI Challenge on Multimodal Brain Tumor Segmentation, Nice, France.
42. Ciçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., and Ronneberger, O. (2016, January 17–21). 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Athens, Greece.10.1007/978-3-319-46723-8_49 ↗
43. Chen, C., Liu, X., Ding, M., Zheng, J., and Li, J. (2019, January 13–17). 3D Dilated Multi-Fiber Network for Real-Timebrain Tumor Segmentation in MRI. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Shenzhen, China.10.1007/978-3-030-32248-9_21 ↗
44. Malmi, E., Parambath, S., Peyrat, J.M., Abinahed, J., and Chawla, S. (2015, January 5–9). Cabs: A Cascaded Brain Tumor Segmentation Approach. Proceedings of the MICCAI Brain, Tumor Segmentation (BRATS), Munich, Germany.
45. Pereira, S., Oliveira, A., Ealves, V., and Silva, C.A. (2017, January 16–18). On Hierarchical Brain Tumor Segmentation in MRI Using Fully Convolutional Neural Networks: A Preliminary Study. Proceedings of the 2017 IEEE 5th Portuguese Meeting on Bioengineering (ENBENG), Coimbra, Portugal.10.1109/ENBENG.2017.7889452 ↗
46. Wang, G., Li, W., Ourselin, S., and Vercauteren, T. (2018, January 11–13). Automatic Brain Tumor Segmentation Using Cascaded Anisotropic Convolutional Neural Networks. Proceedings of the International MICCAI Brain Lesion Workshop, Quebec City, QC, Canada.10.1007/978-3-319-75238-9_16 ↗
47. Liu, H., Shen, X., Shang, F., Ge, F., and Wang, F. (2019, January 17). CU-Net: Cascaded U-Net with Loss Weighted Sampling for Brain Tumor Segmentation. Proceedings of the International Workshop, MFCA 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China.10.1007/978-3-030-33226-6_12 ↗
48. Brügger, R., Baumgartner, C.F., and Konukoglu, E. (2019, January 13–17). A Partially Reversible U-Net for Memory-Efficient Volumetric Image Segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Shenzhen, China.10.1007/978-3-030-32248-9_48 ↗
49. Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., and Maier-Hein, K.H. (2018, January 16). No New-Net. Proceedings of the International MICCAI Brain Lesion Workshop, Granada, Spain.10.1007/978-3-030-11726-9_21 ↗
50. Nuechterlein, N., and Mehta, S. (2018, January 16). 3D-ESPNet with Pyramidal Refinement for Volumetric Brain Tumor Image Segmentation. Proceedings of the International MICCAI Brain Lesion Workshop, Granada, Spain.10.1007/978-3-030-11726-9_22 ↗
51. Kao, P.-Y., Ngo, T., Zhang, A., Chen, J.W., and Manjunath, B.S. (2018, January 16). Brain Tumor Segmentation and Tractographic Feature Extraction from Structural MR Images for Overall Survival Prediction. Proceedings of the International MICCAI Brain Lesion Workshop, Granada, Spain.10.1007/978-3-030-11726-9_12 ↗
52. Chen, W., Liu, B., Peng, S., Sun, J., and Qiao, X. (2019, January 16–20). S3D-UNet: Separable 3D U-Net for Brain Tumor Segmentation. Proceedings of the International MICCAI Brain Lesion Workshop, Granada, Spain.10.1007/978-3-030-11726-9_32 ↗
53. Deep Learning with Mixed Supervision for Brain Tumor Segmentatio10.1117/1.JMI.6.3.034002 ↗
54. Myronenko, A. (2019, January 16–20). 3D MRI Brain Tumor Segmentation Using Autoencoder Regularization. Proceedings of the International MICCAI Brain Lesion Workshop, Granada, Spain.10.1007/978-3-030-11726-9_28 ↗
55. Kampffmeyer, M., Salberg, A.-B., and Jenssen, R. (July, January 26). Semantic Segmentation of Small Objects and Modeling of Uncertainty in Urban Remote Sensing Images Using Deep Convolutional Neural Networks. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Las Vegas, NV, USA.10.1109/CVPRW.2016.90 ↗
56. Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., and Cardoso, M. (2017, January 14). Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations. Proceedings of the International MICCAI Brain Lesion Workshop, Quebec City, QC, Canada.10.1007/978-3-319-67558-9_28 ↗
57. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)10.1109/TMI.2014.2377694 ↗
58. Advancing The Cancer Genome Atlas Glioma MRI Collections With Expert Segmentation Labels and Radiomic Features10.1038/sdata.2017.117 ↗
59. Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., Shinohara, R.T., Berger, C., Ha, S.M., and Rozycki, M. (2018). Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the Brats Challenge. arXiv, Available online: https://arxiv.org/abs/1811.02629.
60. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems, MIT Press.