CASVM: An Efficient Deep Learning Image Classification Method Combined with SVM
Source abstract
Abstract
Recent advances in convolutional neural networks (CNNs) for image feature extraction have achieved extraordinary performance, but back-propagation algorithms tend to fall into local minima. To alleviate this problem, this paper proposes a coordinate attention-support vector machine-convolutional neural network (CASVM). This proposed to enhance the model’s ability by introducing coordinate attention while obtaining enhanced image features. Training is carried out by back-propagating the loss function of support vector machines (SVMs) to improve the generalization capability, which can effectively avoid falling into local optima. The image datasets used in this study for benchmark experiments are Fashion-MNIST, Cifar10, Cifar100, and Animal10. Experimental results show that compared with softmax, CASVM can improve the image classification accuracy of the original model under different image resolution datasets. Under the same structure, CASVM shows better performance and robustness and has higher accuracy. Under the same network parameters, the loss function of CASVM enables the model to realize a lower loss value. Among the standard CNN models, the highest accuracy rate can reach 99%, and the optimal number of accuracy indicators is 5.5 times that of softmax, whose accuracy rate can be improved by up to 56%.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Advanced Image and Video Retrieval Techniques · Domain Adaptation and Few-Shot Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science and Engineering, Chongqing University of Technology, Chongqing 400054, China
Funding
Science and Technology Research Project (Youth) of Chongqing Municipal Education Commission
References (63)
Show source-provided references
1. Support-vector networks10.1007/BF00994018 ↗
2. A comprehensive review on the variants of support vector machines10.1142/S0217984919503032 ↗
3. Tang, Y. (2013). Deep Learning using Linear Support Vector Machines. arXiv.
4. Agarap, A.F.M. (2018, January 26–28). A Neural Network Architecture Combining Gated Recurrent Unit (GRU) and Support Vector Machine (SVM) for Intrusion Detection in Network Traffic Data. Proceedings of the 2018 10th International Conference on Machine Learning and Computing, Macau, China.10.1145/3195106.3195117 ↗
5. Agarap, A.F. (2017). An Architecture Combining Convolutional Neural Network (CNN) and Support Vector Machine (SVM) for Image Classification. arXiv.
6. Deep learning: Methods and applications10.1561/2000000039 ↗
7. Imagenet classification with deep convolutional neural networks
8. Qi, X., Wang, T., and Liu, J. (2017, January 8–10). Comparison of support vector machine and softmax classifiers in computer vision. Proceedings of the 2017 Second International Conference on Mechanical, Control and Computer Engineering (ICMCCE), Harbin, China.10.1109/ICMCCE.2017.49 ↗
9. Survey on SVM and their application in image classification
10. A robust SVM-based approach with feature selection and outliers detection for classification problems10.1016/j.eswa.2021.115017 ↗
11. An Enhancement of Deep Learning Algorithm for Brain Tumor Segmentation Using Kernel Based CNN with M-SVM10.1007/s10916-019-1223-7 ↗
12. Stacking segment-based CNN with SVM for recognition of atrial fibrillation from single-lead ECG recordings10.1016/j.bspc.2021.102672 ↗
13. A hybrid algorithm for lung cancer classification using SVM and Neural Networks10.1016/j.icte.2020.06.007 ↗
14. Speech emotion recognition based on DNN-decision tree SVM model10.1016/j.specom.2019.10.004 ↗
15. A Hybrid CNN-SVM Threshold Segmentation Approach for Tumor Detection and Classification of MRI Brain Images10.1016/j.irbm.2021.06.003 ↗
16. Gong, W., Chen, H., Zhang, Z., Zhang, M., Wang, R., Guan, C., and Wang, Q. (2019). A Novel Deep Learning Method for Intelligent Fault Diagnosis of Rotating Machinery Based on Improved CNN-SVM and Multichannel Data Fusion. Sensors, 19.10.3390/s19071693 ↗
17. Deep learning for finger-knuckle-print identification system based on PCANet and SVM classifier10.1007/s12530-018-9227-y ↗
18. Automated detection of pain levels using deep feature extraction from shutter blinds-based dynamic-sized horizontal patches with facial images10.1038/s41598-022-21380-4 ↗
19. Guo, S., Chen, S., and Li, Y. (2016, January 1–3). Face recognition based on convolutional neural network and support vector machine. Proceedings of the 2016 IEEE International conference on Information and Automation (ICIA), Ningbo, China.10.1109/ICInfA.2016.7832107 ↗
20. Exemplar Darknet19 feature generation technique for automated kidney stone detection with coronal CT images10.1016/j.artmed.2022.102274 ↗
21. Intellectual detection and validation of automated mammogram breast cancer images by multi-class SVM using deep learning classification10.1016/j.imu.2019.01.001 ↗
22. Automated Categorization of Brain Tumor from MRI Using CNN features and SVM10.1007/s12652-020-02568-w ↗
23. Automated BI-RADS classification of lesions using pyramid triple deep feature generator technique on breast ultrasound images10.1016/j.medengphy.2022.103895 ↗
24. Hyperspectral image recognition using SVM combined deep learning
25. Deep support vector machine for hyperspectral image classification10.1016/j.patcog.2020.107298 ↗
26. Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., and Weinberger, K.Q. (2014). Sequence to Sequence Learning with Neural Networks. Advances in Neural Information Processing Systems, Curran Associates, Inc.
27. Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., and Weinberger, K.Q. (2014). Recurrent Models of Visual Attention. Advances in Neural Information Processing Systems, Curran Associates, Inc.
28. A review on the attention mechanism of deep learning10.1016/j.neucom.2021.03.091 ↗
29. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-Excitation Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
30. Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8–14). CBAM: Convolutional Block Attention Module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
31. Gao, Z., Xie, J., Wang, Q., and Li, P. (2019, January 15–20). Global second-order pooling convolutional networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00314 ↗
32. Liu, J.-J., Hou, Q., Cheng, M.-M., Wang, C., and Feng, J. (2020, January 13–19). Improving convolutional networks with self-calibrated convolutions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01011 ↗
33. Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). A^2-Nets: Double Attention Networks. Advances in Neural Information Processing Systems, Curran Associates, Inc.
34. Hou, Q., Zhou, D., and Feng, J. (2021, January 20–25). Coordinate attention for efficient mobile network design. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01350 ↗
35. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
36. Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (2019). Searching for MobileNetV3. arXiv.10.1109/ICCV.2019.00140 ↗
37. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
38. Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018, January 18–23). Learning Transferable Architectures for Scalable Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00907 ↗
39. Tan, M., and Le, Q.V. (2019). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. arXiv.
40. Yavartanoo, M., Hung, S.-H., Neshatavar, R., Zhang, Y., and Lee, K.M. (2021, January 1–3). PolyNet: Polynomial Neural Network for 3D Shape Recognition with PolyShape Representation. Proceedings of the 2021 International Conference on 3D Vision (3DV), London, UK.10.1109/3DV53792.2021.00109 ↗
41. Convolutional support vector machines for speech recognition10.1007/s10772-018-09584-4 ↗
42. Fan, J., Lee, J., and Lee, Y. (2021). A Transfer Learning Architecture Based on a Support Vector Machine for Histopathology Image Classification. Appl. Sci., 11.10.3390/app11146380 ↗
43. Research on diagnosis algorithm of mechanical equipment brake friction fault based on MCNN-SVM10.1016/j.measurement.2021.110065 ↗
44. Franc, V., and Hlavac, V. (2002, January 11–15). Multi-class support vector machine. Proceedings of the 2002 International Conference on Pattern Recognition, Quebec City, QC, Canada.
45. Automated accurate fire detection system using ensemble pretrained residual network10.1016/j.eswa.2022.117407 ↗
46. OAA-SVM-MS: A fast and efficient multi-class classification algorithm10.1016/j.neucom.2021.04.115 ↗
47. A novel kernel-free least squares twin support vector machine for fast and accurate multi-class classification10.1016/j.knosys.2021.107123 ↗
48. Deng, Y., and Deng, Y. (2022). A Method of SAR Image Automatic Target Recognition Based on Convolution Auto-Encode and Support Vector Machine. Remote Sens., 14.10.3390/rs14215559 ↗
49. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv.
50. Ryoo, M.S., Piergiovanni, A., Arnab, A., Dehghani, M., and Angelova, A. (2021). TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?. arXiv.
51. Wortsman, M., Ilharco, G., Gadre, S.Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A.S., Namkoong, H., Farhadi, A., Carmon, Y., and Kornblith, S. (2022, January 17–23). Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. Proceedings of the International Conference on Machine Learning, Baltimore, MD, USA.
52. Zhang, Q., Xu, Y., Zhang, J., and Tao, D. (2022). ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond. arXiv.10.1007/s11263-022-01739-w ↗
53. Huang, T., Huang, L., You, S., Wang, F., Qian, C., and Xu, C. (2022). LightViT: Towards Light-Weight Convolution-Free Vision Transformers. arXiv.
54. Chen, L., Zhang, H., Xiao, J., Nie, L., Shao, J., Liu, W., and Chua, T.-S. (2017, January 21–26). SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.667 ↗
55. Lee, H., Kim, H.-E., and Nam, H. (November, January 27). SRM: A Style-Based Recalibration Module for Convolutional Neural Networks. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.
56. Qin, Z., Zhang, P., Wu, F., and Li, X. (2021, January 10–17). FcaNet: Frequency Channel Attention Networks. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada.10.1109/ICCV48922.2021.00082 ↗
57. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., and Hu, Q. (2020). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. arXiv.10.1109/CVPR42600.2020.01155 ↗
58. Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., and Garnett, R. (2015). Spatial Transformer Networks. Advances in Neural Information Processing Systems, Curran Associates, Inc.
59. Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks. Advances in Neural Information Processing Systems, Curran Associates, Inc.
60. Zhao, H., Zhang, Y., Liu, S., Shi, J., Loy, C.C., Lin, D., and Jia, J. (2018, January 8–14). PSANet: Point-wise Spatial Attention Network for Scene Parsing. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01240-3_17 ↗
61. Zhang, Z., Lan, C., Zeng, W., Jin, X., and Chen, Z. (2020, January 13–19). Relation-Aware Global Attention for Person Re-Identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00325 ↗
62. CNN–SVM: A classification method for fruit fly image with the complex background10.1049/iet-cps.2019.0069 ↗
63. Chaganti, S.Y., Nanda, I., Pandi, K.R., Prudhvith, T.G.N.R.S.N., and Kumar, N. (2020, January 13–14). Image Classification using SVM and CNN. Proceedings of the 2020 International Conference on Computer Science, Engineering and Applications (ICCSEA), Gunupur, India.10.1109/ICCSEA49143.2020.9132851 ↗