journal article·2024-07-16Open access

HLFSRNN-MIL: A Hybrid Multi-Instance Learning Model for 3D CT Image Classification

Huilong Chen, Xiaoxia Zhang

Applied Sciences· Volume 14· Issue 14

Abstract

At present, many diseases are diagnosed by computer tomography (CT) image technology, which affects the health of the lives of millions of people. In the process of disease confrontation, it is very important for patients to detect diseases in the early stage by deep learning of 3D CT images. The paper offers a hybrid multi-instance learning model (HLFSRNN-MIL), which hybridizes high-low frequency feature fusion (HLFFF) with sequential recurrent neural network (SRNN) for CT image classification tasks. Firstly, the hybrid model uses Resnet-50 as the deep feature. The main feature of the HLFSRNN-MIL lies in its ability to make full use of the advantages of the HLFFF and SRNN methods to make up for their own weakness; i.e., the HLFFF can extract more targeted feature information to avoid the problem of excessive gradient fluctuation during training, and the SRNN is used to process the time-related sequences before classification. The experimental study of the HLFSRNN-MIL model is on two public CT datasets, namely, the Cancer Imaging Archive (TCIA) dataset on lung cancer and the China Consortium of Chest CT Image Investigation (CC-CCII) dataset on pneumonia. The experimental results show that the model exhibits better performance and accuracy. On the TCIA dataset, HLFSRNN-MIL with Residual Network (ResNet) as the feature extractor achieves an accuracy (ACC) of 0.992 and an area under curve (AUC) of 0.997. On the CC-CCII dataset, HLFSRNN-MIL achieves an ACC of 0.994 and an AUC of 0.997. Finally, compared with the existing methods, HLFSRNN-MIL has obvious advantages in all aspects. These experimental results demonstrate that HLFSRNN-MIL can effectively solve the disease problem in the field of 3D CT images.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Medical Imaging and Analysis · Medical Image Segmentation Techniques · AI in cancer detection

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan 114051, China

Funding

This work was supported by Innovation and Entrepreneurship Training Program of University of Science and Technology Liaoning (Grant No.X202210146004)

References (45)

Show source-provided references
1. Solving the multiple instance problem with axis-parallel rectangles10.1016/S0004-3702(96)00034-3 ↗
2. Proposing a novel multi-instance learning model for tuberculosis recognition from chest X-ray images based on CNNs, complex networks and stacked ensemble10.1007/s13246-021-00980-w ↗
3. Li, Z., Yuan, L., Xu, H., Cheng, R., and Wen, X. (2020, January 16–19). Deep multi-instance learning with induced self-attention for medical image classification. Proceedings of the 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Seoul, Republic of Korea.10.1109/BIBM49941.2020.9313518 ↗
4. Multiple instance learning with bag dissimilarities10.1016/j.patcog.2014.07.022 ↗
5. MILES: Multiple-instance learning via embedded instance selection10.1109/TPAMI.2006.248 ↗
6. Raykar, V.C., Krishnapuram, B., Bi, J., Dundar, M., and Rao, R.B. (2008, January 5–9). Bayesian multiple instance learning: Automatic feature selection and inductive transfer. Proceedings of the 25th International Conference on Machine Learning 2008, Helsinki, Finland.10.1145/1390156.1390258 ↗
7. Computer-aided diagnosis from weak supervision: A benchmarking study10.1016/j.compmedimag.2014.11.010 ↗
8. Cheplygina, V., Sørensen, L., Tax, D.M.J., de Bruijne, M., and Loog, M. (2015, January 5–9). Label stability in multiple instance learning. Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, 18th International Conference, Munich, Germany. Proceedings, Part I 18.10.1007/978-3-319-24553-9_66 ↗
9. Optimal deep learning model for classification of lung cancer on CT images10.1016/j.future.2018.10.009 ↗
10. 2D-to-3D: A review for computational 3D image reconstruction from X-ray images10.1007/s11831-022-09790-z ↗
11. Deep learning for lung cancer nodules detection and classification in CT scans10.3390/ai1010003 ↗
12. Shi, Z., Mettes, P., Zheng, G., and Snoek, C. (2021). Frequency-supervised mr-to-ct image synthesis. Proceedings of the Deep Generative Models, and Data Augmentation, Labelling, and Imperfections: First Workshop, DGM4MICCAI 2021, and First Workshop, DALI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, 1 October 2021, Springer International Publishing. Proceedings 1.
13. Study on high-precision three-dimensional reconstruction of pulmonary lesions and surrounding blood vessels based on CT images10.1364/OE.510398 ↗
14. Lung cancer identification: A review on detection and classification10.1007/s10555-020-09901-x ↗
15. ImageNet classification with deep convolutional neural networks10.1145/3065386 ↗
16. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
17. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18–22). Shufflenet: An extremely efficient convolutional neural network for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00716 ↗
18. Lung cancer histology classification from CT images based on radiomics and deep learning models10.1007/s11517-020-02302-w ↗
19. Fine-grained lung cancer classification from PET and CT images based on multidimensional attention mechanism10.1155/2020/6153657 ↗
20. Lung-EffNet: Lung cancer classification using EfficientNet from CT-scan images10.1016/j.engappai.2023.106902 ↗
21. Xu, Z., Ren, H., Zhou, W., and Liu, Z. (2022). ISANET: Non-small cell lung cancer classification and detection based on CNN and attention mechanism. Biomed. Signal Process. Control, 77.10.1016/j.bspc.2022.103773 ↗
22. Detection and classification of lung cancer computed tomography images using a novel improved deep belief network with Gabor filters10.1016/j.chemolab.2023.104763 ↗
23. Deep multiple instance learning for predicting chemotherapy response in non-small cell lung cancer using pretreatment CT images10.1038/s41598-022-24278-3 ↗
24. Evaluation of emphysema on thoracic low-dose CTs through attention-based multiple instance deep learning10.1038/s41598-023-27549-9 ↗
25. Tang, W., Huang, S., Zhang, X., Zhou, F., Zhang, Y., and Liu, B. (2023, January 2–3). Multiple instance learning framework with masked hard instance mining for whole slide image classification. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00377 ↗
26. Bilateral adaptive graph convolutional network on CT based Covid-19 diagnosis with uncertainty-aware consensus-assisted multiple instance learning10.1016/j.media.2022.102722 ↗
27. Xue, M., Jia, S., Chen, L., Huang, H., Yu, L., and Zhu, W. (2023). CT-based COPD identification using multiple instance learning with two-stage attention. Comput. Methods Programs Biomed., 230.10.1016/j.cmpb.2023.107356 ↗
28. Deep semi-supervised multiple instance learning with self-correction for DME classification from OCT images10.1016/j.media.2022.102673 ↗
29. Advmil: Adversarial multiple instance learning for the survival analysis on whole-slide images10.1016/j.media.2023.103020 ↗
30. Schmidt, A., Morales-Alvarez, P., and Molina, R. (2023). Probabilistic attention based on gaussian processes for deep multiple instance learning. IEEE Trans. Neural Netw. Learn. Syst.10.1109/TNNLS.2023.3245329 ↗
31. Wang, H., Luo, L., Wang, F., Tong, R., Chen, Y.W., Hu, H., and Chen, H. (2023). Iteratively coupled multiple instance learning from instance to bag classifier for whole slide image classification. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer Nature.10.1007/978-3-031-43987-2_45 ↗
32. Causality-driven graph neural network for early diagnosis of pancreatic cancer in non-contrast computerized tomography10.1109/TMI.2023.3236162 ↗
33. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
34. Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., and Adam, H. (November, January 27). Searching for mobilenetv3. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
35. Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., and Jégou, H. (2021, January 11–17). Going deeper with image transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00010 ↗
36. Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography10.1016/j.cell.2020.04.045 ↗
37. Attention-based Deep Multiple Instance Learning
38. Ilse, M., Tomczak, J.M., and Welling, M. (2020). Deep multiple instance learning for digital histopathology. Handbook of Medical Image Computing and Computer Assisted Intervention, Academic Press.10.1016/B978-0-12-816176-0.00027-2 ↗
39. Lung cancer diagnosis using deep attention-based multiple instance learning and radiomics10.1002/mp.15539 ↗
40. Fast vision transformers with hilo attention
41. Context-aware multi-head self-attentional neural network model for next location prediction10.1016/j.trc.2023.104315 ↗
42. Qi, S., Xu, C., Li, C., Tian, B., Xia, S., Ren, J., and Yu, H. (2021). DR-MIL: Deep represented multiple instance learning distinguishes COVID-19 from community-acquired pneumonia in CT images. Comput. Methods Programs Biomed., 211.10.1016/j.cmpb.2021.106406 ↗
43. Explainable multi-instance and multi-task learning for COVID-19 diagnosis and lesion segmentation in CT images10.1016/j.knosys.2022.109278 ↗
44. Arsenos, A., Kollias, D., and Kollias, S. (2022, January 26–29). A large imaging database and novel deep neural architecture for COVID-19 diagnosis. Proceedings of the 2022 IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP), Nafplio, Greece.10.1109/IVMSP54334.2022.9816321 ↗
45. Robust weakly supervised learning for COVID-19 recognition using multi-center CT images10.1016/j.asoc.2021.108291 ↗