High Inclusiveness and Accuracy Motion Blur Real-Time Gesture Recognition Based on YOLOv4 Model Combined Attention Mechanism and DeblurGanv2
Source abstract
Abstract
The combination of gesture recognition and aerospace exploration robots can realize the efficient non-contact control of the robots. In the harsh aerospace environment, the captured gesture images are usually blurred and damaged inevitably. The motion blurred images not only cause part of the transmitted information to be lost, but also affect the effect of neural network training in the later stage. To improve the speed and accuracy of motion blurred gestures recognition, the algorithm of YOLOv4 (You Only Look Once, vision 4) is studied from the two aspects of motion blurred image processing and model optimization. The DeblurGanv2 is employed to remove the motion blur of the gestures in YOLOv4 network input pictures. In terms of model structure, the K-means++ algorithm is used to cluster the priori boxes for obtaining the more appropriate size parameters of the priori boxes. The CBAM attention mechanism and SPP (spatial pyramid pooling layer) structure are added to YOLOv4 model to improve the efficiency of network learning. The dataset for network training is designed for the human–computer interaction in the aerospace space. To reduce the redundant features of the captured images and enhance the effect of model training, the Wiener filter and bilateral filter are superimposed on the blurred images in the dataset to simply remove the motion blur. The augmentation of the model is executed by imitating different environments. A YOLOv4-gesture model is built, which collaborates with K-means++ algorithm, the CBAM and SPP mechanism. A DeblurGanv2 model is built to process the input images of the YOLOv4 target recognition. The YOLOv4-motion-blur-gesture model is composed of the YOLOv4-gesture and the DeblurGanv2. The augmented and enhanced gesture data set is used to simulate the model training. The experimental results demonstrate that the YOLOv4-motion-blur-gesture model has relatively better performance. The proposed model has the high inclusiveness and accuracy recognition effect in the real-time interaction of motion blur gestures, it improves the network training speed by 30%, the target detection accuracy by 10%, and the value of mAP by about 10%. The constructed YOLOv4-motion-blur-gesture model has a stable performance. It can not only meet the real-time human–computer interaction in aerospace space under real-time complex conditions, but also can be applied to other application environments under complex backgrounds requiring real-time detection.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Image Processing Techniques and Applications · Image and Video Stabilization · Advanced Image Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechanical Engineering, Tianjin University of Technology and Education, Tianjin 300222, China
School of Information Technology Engineering, Tianjin University of Technology and Education, Tianjin 300222, China
Funding
National Natural Science Foundation of China · Doctor Startup Projects of TUTE
References (31)
Show source-provided references
1. Quickly obtaining range of articulated rotating speed for electrically driven large-load-ratio six-legged robot based on maximum walking speed method10.1109/ACCESS.2019.2901903 ↗
2. Zhuang, H.C., Gao, H.B., and Deng, Z.Q. (2017). Gait planning research for an electrically driven large-load-ratio six-legged robot. Appl. Sci., 7.10.3390/app7030296 ↗
3. Bian, F., Li, R., and Liang, P. (2017, January 6–9). SVM based simultaneous hand movements classification using sEMG signals. Proceedings of the 2017 IEEE International Conference on Mechatronics and Automation (ICMA), Takamatsu, Japan.10.1109/ICMA.2017.8015855 ↗
4. Comparative analysis among feature selection of sEMG signal for hand gesture classification by armband10.1109/TLA.2020.9099752 ↗
5. SIFT: Predicting amino acid changes that affect protein function10.1093/nar/gkg509 ↗
6. Wang, X., Han, T.X., and Yan, S. (October, January 27). An HOG-LBP human detector with partial occlusion handling. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.10.1109/ICCV.2009.5459207 ↗
7. Mistry, P., Maes, P., and Chang, L. (2009, January 4–9). WUW-wear Ur world: A wearable gestural interface. Proceedings of the 27th International Conference on Human Factors in Computing Systems, Boston, MA, USA.10.1145/1520340.1520626 ↗
8. Wilhelm, M., Krakowczyk, D., and Albayrak, S. (2020). PeriSense: Ring-Based Multi-Finger Gesture Interaction Utilizing Capacitive Proximity Sensing. Sensors, 20.10.3390/s20143990 ↗
9. A novel, co-located EMG-FMG-sensing wearable armband for hand gesture recognition10.1016/j.sna.2019.111738 ↗
10. A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition10.1038/s41928-020-00510-8 ↗
11. Pneumatic actuator and flexible piezoelectric sensor for soft virtual reality glove system10.1038/s41598-019-45422-6 ↗
12. 3D motion tracking display enabled by magneto-interactive electroluminescence10.1038/s41467-020-19523-0 ↗
13. Mantecón, T., Del-Blanco, C.R., Jaureguizar, F., and García, N. (2016, January 24–27). Hand gesture recognition using infrared imagery provided by leap motion controller. Proceedings of the International Conference on Advanced Concepts for Intelligent Vision Systems, Lecce, Italy.10.1007/978-3-319-48680-2_5 ↗
14. Dermatologist-level classification of skin cancer with deep neural networks10.1038/nature21056 ↗
15. Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors10.1038/s41928-020-0422-z ↗
16. YOLO based human action recognition and localization10.1016/j.procs.2018.07.112 ↗
17. Yu, J.M., and Zhang, W. (2021). Face mask wearing detection algorithm based on improved YOLO-v4. Sensors, 21.10.3390/s21093263 ↗
18. A Deep Learning Enabled Multi-Class Plant Disease Detection Model Based on Computer Vision10.3390/ai2030026 ↗
19. MultiD-CNN: A multi-dimensional feature learning approach based on deep convolutional networks for gesture recognition in RGB-D image sequences10.1016/j.eswa.2019.112829 ↗
20. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.
21. Gao, M.Y., Cai, Q.Y., Zheng, B.W., Shi, J., Ni, Z.H., Wang, J.F., and Lin, H.P. (2021). A Hybrid YOLO v4 and Particle Filter Based Robotic Arm Grabbing System in Nonlinear and Non-Gaussian Environment. Electronics, 10.10.3390/electronics10101140 ↗
22. Huang, R., Pedoeem, J., and Chen, C.X. (2018, January 10–13). YOLO-LITE: A real-time object detection algorithm optimized for non-GPU computers. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.10.1109/BigData.2018.8621865 ↗
23. Algorithm AS 136: A k-means clustering algorithm
24. Arthur, D., and Vassilvitskii, S. (2007, January 7–9). K-means ++: The advantages of carefull seeding. Proceedings of the Eighteenth Annual ACM-SIAM Symposiumon Discrete Algorithms, New Orleans, LA, USA.
25. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV) 2018, Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
26. Enhanced yolo v3 tiny network for real-time ship detection from visual image10.1109/ACCESS.2021.3053956 ↗
27. Spatial pyramid pooling in deep convolutional networks for visual recognition10.1109/TPAMI.2015.2389824 ↗
28. Kupyn, O., Martyniuk, T., Wu, J.R., and Wang, Z.Y. (November, January 27). Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.
29. A fast image super-resolution algorithm using an adaptive Wiener filter10.1109/TIP.2007.909416 ↗
30. Multiresolution bilateral filtering for image denoising10.1109/TIP.2008.2006658 ↗
31. Deep learning in remote sensing scene classification: A data augmentation enhanced convolutional neural network framework10.1080/15481603.2017.1323377 ↗