Lightweight Human Ear Recognition Based on Attention Mechanism and Feature Fusion
Source abstract
Abstract
With the development of deep learning technology, more and more researchers are interested in ear recognition. Human ear recognition is a biometric identification technology based on human ear feature information and it is often used for authentication and intelligent monitoring field, etc. In order to make ear recognition better applied to practical application, real time and accuracy have always been very important and challenging topics. Therefore, focusing on the problem that the mAP@0.5 value of the YOLOv5s-MG method is lower than that of the YOLOv5s method on the EarVN1.0 human ear dataset with low resolution, small target, rotation, brightness change, earrings, glasses and other occlusion, a lightweight ear recognition method is proposed based on an attention mechanism and feature fusion. This method mainly includes the following several steps: First, the CBAM attention mechanism is added to the connection between the backbone network and the neck network of the lightweight human ear recognition method YOLOv5s-MG, and the YOLOv5s-MG-CBAM human ear recognition network is constructed, which can improve the accuracy of the method. Second, the SPPF layer and cross-regional feature fusion are added to construct the YOLOv5s-MG-CBAM-F human ear recognition method, which further improves the accuracy. Three distinctive human ear datasets, namely, CCU-DE, USTB and EarVN1.0, are used to evaluate the proposed method. Through the experimental comparison of seven methods including YOLOv5s-MG-CBAM-F, YOLOv5s-MG-SE-F, YOLOv5s-MG-CA-F, YOLOv5s-MG-ECA-F, YOLOv5s, YOLOv7 and YOLOv5s-MG on the EarVN1.0 human ear dataset, it is found that the human ear recognition rate of YOLOv5s-MG-CBAM-F method is the highest. The mAP@0.5 value of the proposed YOLOv5s-MG-CBAM-F method on the EarVN1.0 ear dataset is 91.9%, which is 6.4% higher than that of the YOLOv5s-MG method and 3.7% higher than that of the YOLOv5s method. The params, GFLOPS, model size and the inference time per image of YOLOv5s-MG-CBAM-F method on the EarVN1.0 human ear dataset are 5.2 M, 8.3 G, 10.9 MB and 16.4 ms, respectively, which are higher than the same parameters of the YOLOv5s-MG method, but less than the same parameters of YOLOv5s method. The quantitative results show that the proposed method can improve the ear recognition rate while satisfying the real-time performance and it is especially suitable for applications where high ear recognition rates are required.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Biometric Identification and Security · Face recognition and analysis · Reconstructive Facial Surgery Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical and Information Engineering, Changchun University, Changchun 130022, China
Institute of Science and Technology, Changchun Humanities and Sciences College, Changchun 130028, China
Aviation Basic College, Air Force Aviation University, Changchun 130022, China
Jilin Province Key Laboratory of Measuring Instrument and Technology, Jilin Institute of Metrology, Changchun 130103, China
Funding
Education Department of Jilin Province
References (26)
Show source-provided references
1. Oyebiyi, O.G., Abayomi-Alli, A., Arogundade, O.T., Qazi, A., Imoize, A.L., and Awotunde, J.B. (2023). A Systematic Literature Review on Human Ear Biometrics: Approaches, Algorithms, and Trend in the LastDecade. Information, 14.10.3390/info14030192 ↗
2. Zhang, Y., and Mu, Z. (2017). Ear Detection under Uncontrolled Conditions with Multiple Scale Faster Region-Based Convolutional Neural Networks. Symmetry, 9.10.3390/sym9040053 ↗
3. Fan, T.Y., Mu, Z.C., and Yang, R.Y. (2017, January 9–12). Multi-modality recognition of human face and ear based on deep learning. Proceedings of the 2017 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR), Ningbo, China.10.1109/ICWAPR.2017.8076660 ↗
4. A comprehensive survey and deep learning-based approach for human recognition using ear biometric10.1007/s00371-021-02119-0 ↗
5. CED-Net: Context-aware ear detection network for unconstrained images10.1007/s10044-020-00914-4 ↗
6. Qian, J.R. (2020). Research on Dynamic Human Ear Recognition Method Based on deep Learning. [Master’s Thesis, Chang Chun University].
7. Quoc, H.N., and Hoang, V.T. (2021, January 7–8). Human ear-side detection based on YOLOv5 detector and deep neural networks. Proceedings of the 2021 International Conference on Decision Aid Sciences and Application (DASA), Sakheer, Bahrain.10.1109/DASA53625.2021.9682306 ↗
8. Tian, Y., Wang, S., and Li, W. (2018, January 9–11). Human ear recognition based on deep convolutional neural network. Proceedings of the 2018 Chinese Control and Decision Conference (CCDC), Shenyang, China.
9. Mehraj, H., and Mir, A.H. (2020, January 12–14). Human Recognition using Ear based Deep Learning Features. Proceedings of the 2020 International Conference on Emerging Smart Computing and Informatics (ESCI), Pune, India.10.1109/ESCI48226.2020.9167641 ↗
10. Dvoršak, G., Dwivedi, A., Štruc, V., Peer, P., and Emeršič, Ž. (2022, January 20–21). Kinship Verification from Ear Images: An Explorative Study with Deep Learning Models. Proceedings of the 2022 International Workshop on Biometrics and Forensics (IWBF), Salzburg, Austria.10.1109/IWBF55382.2022.9794555 ↗
11. Image Quality Assessment for Effective Ear Recognition10.1109/ACCESS.2022.3206024 ↗
12. Exploring Deep Learning Ear Recognition in Thermal Images10.1109/TBIOM.2022.3218151 ↗
13. Yuan, L., Mao, J., and Zheng, H. (2020, January 14–16). Ear Detection based on CenterNet. Proceedings of the 2020 IEEE 2nd International Conference on Civil Aviation Safety and Information Technology (ICCASIT), Weihai, China.10.1109/ICCASIT50869.2020.9368856 ↗
14. Deep Learning for 3D Ear Detection: A Complete Pipeline from Data Generation to Segmentation10.1109/ACCESS.2021.3129507 ↗
15. Mursalin, M., Ahmed, M., and Haskell-Dowland, P. (2022). Biometric Security: A Novel Ear Recognition Approach Using a 3D Morphable Ear Model. Sensors, 22.10.3390/s22228988 ↗
16. A hybrid approach based on deep learning for gender recognition using human ear images
17. Lei, Y., Pan, D., Feng, Z., and Qian, J. (2023). Lightweight YOLOv5s Human Ear Recognition Based on MobileNetV3 and Ghostnet. Appl. Sci., 13.10.3390/app13116667 ↗
18. Squeeze-and-Excitation Networks
19. Hou, Q., Zhou, D., and Feng, J. (2021, January 20–25). Coordinate Attention for Efficient Mobile Network Design. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01350 ↗
20. Wang, Q., Wu, B., and Zhu, P. (2020, January 13–9). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01155 ↗
21. Woo, S., Park, J., and Lee, J.Y. (2020, January 23–28). CBAM: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK.
22. Liu, S., Qi, L., and Qin, H. (2018, January 18–23). Path Aggregation Network for Instance Segmentation. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00913 ↗
23. Lin, T.Y., Dollar, P., and Girshick, R. (2017, January 21–26). Feature Pyramid Networks for Object Detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
24. Plant disease and pest detection based on vision attention enhancement
25. Improved YOLOv5 small object detection algorithm in moving scenes
26. A Lightweight Object Detection Algorithm Based on Improved YOLOv5s