A Histopathological Image Classification Method Based on Model Fusion in the Weight Space
Source abstract
Abstract
Automatic classification of histopathological images plays an important role in computer-aided diagnosis systems. The automatic classification model of histopathological images based on deep neural networks has received widespread attention. However, the performance of deep models is affected by many factors, such as training hyperparameters, model structure, dataset quality, and training cost. In order to reduce the impact of the above factors on model training and reduce the training and inference costs of the model, we propose a novel method based on model fusion in the weight space, which is inspired by stochastic weight averaging and model soup. We use the cyclical learning rate (CLR) strategy to fine-tune the ingredient models and propose a ranking strategy based on accuracy and diversity for candidate model selection. Compared to the single model, the weight fusion of ingredient models can obtain a model whose performance is closer to the expected value of the error basin, which may improve the generalization ability of the model. Compared to the ensemble model with n base models, the testing cost of the proposed model is theoretically 1/n of that of the ensemble model. Experimental results on two histopathological image datasets show the effectiveness of the proposed model in comparison to baseline ones, including ResNet, VGG, DenseNet, and their ensemble versions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: AI in cancer detection · Digital Imaging for Blood Diseases · Medical Imaging and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Automation, Guangdong University of Technology, Guangzhou 510006, China
Information Engineering College, Guangzhou Panyu Polytechnic, Guangzhou 511483, China
School of Computer Science, Guangdong University of Education, Guangzhou 510303, China
Funding
National Natural Science Foundation of China · Guangzhou City and the School · Science and Technology Projects in Guangzhou · Key Platforms and Scientific Research Projects in Universities in Guangdong Province
References (40)
Show source-provided references
1. Validation of digital pathology imaging for primary histopathological diagnosis10.1111/his.12879 ↗
2. Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images10.1016/j.celrep.2018.03.086 ↗
3. AI in medical imaging informatics: Current challenges and future directions10.1109/JBHI.2020.2991043 ↗
4. Deep learning10.1038/nature14539 ↗
5. Shinde, P.P., and Shah, S. (2018, January 16–18). A review of machine learning and deep learning applications. Proceedings of the 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), Pune, India.10.1109/ICCUBEA.2018.8697857 ↗
6. Deep learning in bioinformatics: Introduction, application, and perspective in the big data era10.1016/j.ymeth.2019.04.008 ↗
7. A Series-Based Deep Learning Approach to Lung Nodule Image Classification10.3390/cancers15030843 ↗
8. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
9. Dai, Z., Liu, H., Le, Q.V., and Tan, M. (2021). CoAtNet: Marrying Convolution and Attention for All Data Sizes. arXiv.
10. Accurate patient-specific machine learning models of glioblastoma invasion using transfer learning
11. Detection and classification of novel renal histologic phenotypes using deep neural networks10.1016/j.ajpath.2019.05.019 ↗
12. Roster, K., Connaughton, C., and Rodrigues, F.A. (2022). Forecasting new diseases in low-datasettings using transfer learning. arXiv.10.1016/j.chaos.2022.112306 ↗
13. Histologic tissue components provide major cues for machine learning-based prostate cancer detection and grading on prostatectomy specimens10.1038/s41598-020-66849-2 ↗
14. Breast cancer masses classification using deep convolutional neural networks and transfer learning10.1007/s11042-020-09518-w ↗
15. Liu, W., Mo, J., and Zhong, F. (2023). Class Imbalanced Medical Image Classification Based on Semi-Supervised Federated Learning. Appl. Sci., 13.10.3390/app13042109 ↗
16. Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D., and Wilson, A.G. (2018). Averaging weights leads to wider optima and better generalization. arXiv.
17. Wortsman, M., Ilharco, G., Gadre, S.Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A.S., Namkoong, H., Farhadi, A., Carmon, Y., and Kornblith, S. (2022). Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. arXiv.
18. Choshen, L., Venezian, E., Slonim, N., and Katz, Y. (2022). Fusing finetuned models for better pretraining. arXiv.
19. Dansereau, C., Sobral, M., Bhogal, M., and Zalai, M. (2023). Model soups to increase inference without increasing compute time. arXiv.
20. Draxler, F., Veschgini, K., Salmhofer, M., and Hamprecht, F. (2018, January 10–15). Essentially no barriers in neural network energy landscape. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden.
21. Smith, L.N. (2017, January 24–31). Cyclical learning rates for training neural networks. Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA.10.1109/WACV.2017.58 ↗
22. A novel hierarchical selective ensemble classifier with bioinformatics application10.1016/j.artmed.2017.02.005 ↗
23. Smith, L.N., and Topin, N. (2017). Exploring loss function topology with cyclical learning rates. arXiv.
24. Garipov, T., Izmailov, P., Podoprikhin, D., Vetrov, D.P., and Wilson, A.G. (2018, January 3–8). Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs. Proceedings of the Advances in Neural Information Processing Systems 31 (NeurIPS 2018), Montreal, QC, Canada.
25. Parallelizing Stochastic Gradient Descent for Least Squares Regression: Mini-batching, Averaging, and Model Misspecification
26. Guo, H., Jin, J., and Liu, B. (2023). Stochastic Weight Averaging Revisited. Appl. Sci., 13.10.3390/app13052935 ↗
27. What is being transferred in transfer learning?
28. Hameed, Z., Zahia, S., Garcia-Zapirain, B., Javier Aguirre, J., and María Vanegas, A. (2020). Breast cancer histopathology image classification using an ensemble of deep learning models. Sensors, 20.10.3390/s20164373 ↗
29. Mitotic nuclei analysis in breast cancer histopathology images using deep ensemble classifier10.1016/j.media.2021.102121 ↗
30. Classification of Invasive Ductal Carcinoma from histopathology breast cancer images using Stacked Generalized Ensemble10.3233/JIFS-201702 ↗
31. Deep transfer learning-based automated detection of COVID-19 from lung CT scan slices10.1007/s10489-020-01826-w ↗
32. Detection of COVID-19 from CT Lung Scans Using Transfer Learning10.1155/2021/5527923 ↗
33. Automatic detection of COVID-19 from chest CT scan and chest X-Rays images using deep learning, transfer learning and stacking10.1007/s10489-021-02393-4 ↗
34. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
35. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
36. Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densenet: Densely connected convolutional networks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
37. Cubuk, E.D., Zoph, B., Shlens, J., and Le, Q.V. (2020, January 14–19). RandAugment: Practical automated data augmentation with a reduced search space. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00359 ↗
38. Lu, Z., Wu, X., Zhu, X., and Bongard, J. (2010, January 25–28). Ensemble pruning via individual contribution ordering. Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, DC, USA.10.1145/1835804.1835914 ↗
39. Dual image-based reversible fragile watermarking scheme for tamper detection and localization10.1007/s10044-022-01104-0 ↗
40. Statistical Comparisons of Classifiers over Multiple Data Sets