MedMamba-ESS: A Generalizable State Space Framework for Clinical Image Classification with Enhanced Performance and Computational Efficiency
Source abstract
Abstract
Since deep learning has been applied to medical image analysis, convolutional neural networks (CNNs) and Vision Transformers (ViTs) have become core architectures for medical image classification tasks. However, CNNs are constrained by local receptive fields and cannot effectively model long-range context-dependent correlations in medical images; ViTs face deployment challenges in high-resolution medical image tasks due to the quadratic computational complexity of self-attention. While State Space Models (SSMs) represented by Mamba offer a new solution with linear complexity, they suffer from redundant directional modeling and low parameter efficiency in direct medical image applications. This study proposes the efficient state space enhancement framework (MedMamba-ESS), integrating the SS2D-Top2 adaptive directional scanning mechanism (reducing SSM submodule FLOPs by ~37%) and the G-SSM grouped parameter sharing module (achieving 3–4% parameter compression and 4.4% accuracy improvement via regularization). Validated on 14 public datasets (9 imaging modalities, 9 anatomical regions, >240,000 images), MedMamba-ESS achieves superior performance at 2.0G FLOPs: its Overall Accuracy (OA) is 3% higher than the baseline MedMamba-Tiny on non-MedMNIST datasets (3.13% higher than other mainstream models) and 3% higher on MedMNIST datasets (1% higher than others). Ablation experiments confirm that the two modules synergistically reduce parameters by 1.05% and boost accuracy by 4.6%. This study overcomes the technical limitations of traditional SSMs in medical imaging applications, achieving synergistic improvements in both model performance and computational efficiency. It provides an architecture optimization solution that combines practicality and generalizability for the implementation of SSMs in medical image analysis.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Machine Learning in Healthcare · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Information Engineering, Sichuan Agricultural University, Ya’an 625014, China
College of Mechanical and Electrical Engineering, Sichuan Agricultural University, Ya’an 625014, China
Funding
Jiong Mu
References (56)
Show source-provided references
1. Deep convolutional neural network based medical image classification for disease diagnosis10.1186/s40537-019-0276-2 ↗
2. Preventing corneal blindness caused by keratitis using artificial intelligence10.1038/s41467-021-24116-6 ↗
3. Medical Imaging: From Roentgen to the Digital Revolution, and Beyond10.5041/RMMJ.10355 ↗
4. Li, M., Jiang, Y., Zhang, Y., and Zhu, H. (2023). Medical image analysis using deep learning algorithms. Front. Public. Health, 11.10.3389/fpubh.2023.1273253 ↗
5. Winder, M., Owczarek, A.J., Chudek, J., Pilch-Kowalczyk, J., and Baron, J. (2021). Are We Overdoing It? Changes in Diagnostic Imaging Workload during the Years 2010–2020 including the Impact of the SARS-CoV-2 Pandemic. Healthcare, 9.10.3390/healthcare9111557 ↗
6. Inpatient Imaging Utilization and Radiology Workload: Trends of the Past Decade and Through the COVID-19 Pandemic10.1016/j.jacr.2025.05.011 ↗
7. Cognitive and System Factors Contributing to Diagnostic Errors in Radiology10.2214/AJR.12.10375 ↗
8. Interobserver variability studies in diagnostic imaging: A methodological systematic review10.1259/bjr.20220972 ↗
9. A survey on deep learning in medical image analysis10.1016/j.media.2017.07.005 ↗
10. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv.
11. A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises10.1109/JPROC.2021.3054390 ↗
12. Vaswani, A., Shazeer, N.M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention is All you Need. Proceedings of the Neural Information Processing Systems 2017, Long Beach, CA, USA.
13. Transformers in medical imaging: A survey10.1016/j.media.2023.102802 ↗
14. Gu, A., and Dao, T. (2023). Mamba: Linear-Time Sequence Modeling with Selective State Spaces. arXiv.
15. Gu, A., Johnson, I., Goel, K., Saab, K.K., Dao, T., Rudra, A., and R’e, C. (2021, January 6–14). Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers. Proceedings of the Neural Information Processing Systems 2021, Virtual.
16. Maudet, G., and Danoy, G. (2025). A Distance Metric for Mixed Integer Programming Instances. arXiv.10.3233/FAIA250894 ↗
17. Dang, T., Nguyen, H.H., and Tiulpin, A. (2024). LoG-VMamba: Local-Global Vision Mamba for Medical Image Segmentation. arXiv.10.1007/978-981-96-0901-7_14 ↗
18. Yan, X., Jiang, W., Shi, Y., and Zhuo, C. (2020, January 4–8). MS-NAS: Multi-scale Neural Architecture Search for Medical Image Segmentation. Proceedings of the Medical Image Computing and Computer Assisted Intervention—MICCAI 2020, Lima, Peru.10.1007/978-3-030-59710-8_38 ↗
19. Multimodal Machine Learning: A Survey and Taxonomy
20. Yue, Y., and Li, Z. (2024). MedMamba: Vision Mamba for Medical Image Classification. arXiv.
21. PAD-UFES-20: A skin lesion dataset composed of patient data and clinical images collected from smartphones10.1016/j.dib.2020.106221 ↗
22. CheXImageNet: A novel architecture for accurate classification of Covid-19 with chest x-ray digital images using deep convolutional neural networks10.1007/s12553-021-00630-x ↗
23. LiteCovidNet: A lightweight deep neural network model for detection of COVID-19 using X-ray images10.1002/ima.22770 ↗
24. Evaluation of deep convolutional neural networks for automatic classification of common maternal fetal ultrasound planes10.1038/s41598-020-67076-5 ↗
25. Pogorelov, K., Randel, K.R., Griwodz, C., Eskeland, S.L., Lange, T.D., Johansen, D., Spampinato, C., Dang-Nguyen, D.-T., Lux, M., and Schmidt, P.T. (2017, January 20–23). KVASIR: A Multi-Class Image Dataset for Computer Aided Gastrointestinal Disease Detection. Proceedings of the 8th ACM on Multimedia Systems Conference, Taipei, Taiwan.10.1145/3083187.3083212 ↗
26. MedMNIST v2—A large-scale lightweight benchmark for 2D and 3D biomedical image classification10.1038/s41597-022-01721-8 ↗
27. Yang, J., Shi, R., and Ni, B. (2021, January 13–16). MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis. Proceedings of the 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), Nice, France.10.1109/ISBI48211.2021.9434062 ↗
28. MedViT: A robust vision transformer for generalized medical image classification10.1016/j.compbiomed.2023.106791 ↗
29. Liu, Y., Tian, Y., Zhao, Y., Yu, H., Xie, L., Wang, Y., Ye, Q., and Liu, Y. (2024). VMamba: Visual State Space Model. arXiv.
30. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
31. Hua, W., Dai, Z., Liu, H., and Le, Q.V. (2022, January 17–23). Transformer Quality in Linear Time. Proceedings of the International Conference on Machine Learning 2022, Baltimore, MD, USA.
32. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10–17). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00986 ↗
33. Liu, Z., Mao, H., Wu, C., Feichtenhofer, C., Darrell, T., and Xie, S. (2022, January 18–24). A ConvNet for the 2020s. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01167 ↗
34. MetaFormer Baselines for Vision10.1109/TPAMI.2023.3329173 ↗
35. Han, K., Xiao, A., Wu, E., Guo, J., Xu, C., and Wang, Y. (2021, January 6–14). Transformer in Transformer. Proceedings of the Neural Information Processing Systems 2021, Virtual.
36. PVT v2: Improved baselines with Pyramid Vision Transformer10.1007/s41095-022-0274-8 ↗
37. Ding, M., Xiao, B., Codella, N.C.F., Luo, P., Wang, J., and Yuan, L. (2022, January 23–27). DaViT: Dual Attention Vision Transformers. Proceedings of the European Conference on Computer Vision 2022, Tel Aviv, Israel.10.1007/978-3-031-20053-3_5 ↗
38. Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and J’egou, H.E. (2020, January 13–18). Training data-efficient image transformers & distillation through attention. Proceedings of the International Conference on Machine Learning 2020, Virtual.
39. Tan, M., and Le, Q.V. (2019). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. arXiv.
40. Xu, W., Xu, Y., Chang, T.A., and Tu, Z. (2021, January 10–17). Co-Scale Conv-Attentional Image Transformers. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00983 ↗
41. Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., and Sun, J. (2021, January 20–25). RepVGG: Making VGG-style ConvNets Great Again. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01352 ↗
42. Mehta, S., and Rastegari, M. (2022). Separable Self-attention for Mobile Vision Transformers. arXiv.
43. Maaz, M., Shaker, A.M., Cholakkal, H., Khan, S.H., Zamir, S.W., Anwer, R.M., and Khan, F.S. (2022, January 23–27). EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications. Proceedings of the ECCV Workshops 2022, Tel Aviv, Israel.10.1007/978-3-031-25082-8_1 ↗
44. Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding
45. Anasosalu Vasu, P.K., Gabriel, J.G., Zhu, J.J., Tuzel, O., and Ranjan, A. (2023, January 17–24). MobileOne: An Improved One millisecond Mobile Backbone. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.
46. Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., and J’egou, H.E. (2021, January 10–17). Going deeper with Image Transformers. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00010 ↗
47. Li, Y., Wu, C., Fan, H., Mangalam, K., Xiong, B., Malik, J., and Feichtenhofer, C. (2022, January 18–24). MViTv2: Improved Multiscale Vision Transformers for Classification and Detection. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00476 ↗
48. Yang, J., Li, C., and Gao, J. (2022). Focal Modulation Networks. arXiv.
49. Chu, X., Tian, Z., Wang, Y., Zhang, B., Ren, H., Wei, X., Xia, H., and Shen, C. (2021, January 6–14). Twins: Revisiting the Design of Spatial Attention in Vision Transformers. Proceedings of the Neural Information Processing Systems 2021, Virtual.
50. Yu, W., Luo, R.M., Zhou, P., Si, C., Zhou, Y., Wang, X., Feng, J., and Yan, S. (2022, January 18–24). MetaFormer is Actually What You Need for Vision. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01055 ↗
51. El-Nouby, A., Touvron, H., Caron, M., Bojanowski, P., Douze, M., Joulin, A., Laptev, I., Neverova, N., Synnaeve, G., and Verbeek, J. (2021, January 6–14). XCiT: Cross-Covariance Image Transformers. Proceedings of the Neural Information Processing Systems 2021, Virtual.
52. Hatamizadeh, A., Yin, H., Kautz, J., and Molchanov, P. (2022, January 17–23). Global Context Vision Transformers. Proceedings of the International Conference on Machine Learning 2022, Baltimore, MD, USA.
53. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
54. Hutter, F., Kotthoff, L., and Vanschoren, J. (2019). Auto-sklearn: Efficient and Robust Automated Machine Learning. Automated Machine Learning: Methods, Systems, Challenges, Springer International Publishing.10.1007/978-3-030-05318-5 ↗
55. Jin, H., Song, Q., and Hu, X. (2019, January 4–8). Auto-Keras: An Efficient Neural Architecture Search System. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.10.1145/3292500.3330648 ↗
56. Bisong, E. (2019). Building Machine Learning and Deep Learning Models on Google Cloud Platform: A Comprehensive Guide for Beginners, Apress.10.1007/978-1-4842-4470-8 ↗