Gesture Recognition and Hand Tracking for Anti-Counterfeit Palmvein Recognition
Source abstract
Abstract
At present, COVID-19 is posing a serious threat to global human health. The features of hand veins in infrared environments have many advantages, including non-contact acquisition, security, privacy, etc., which can remarkably reduce the risks of COVID-19. Therefore, this paper builds an interactive system, which can recognize hand gestures and track hands for palmvein recognition in infrared environments. The gesture contours are extracted and input into an improved convolutional neural network for gesture recognition. The hand is tracked based on key point detection. Because the hand gesture commands are randomly generated and the hand vein features are extracted from the infrared environment, the anti-counterfeiting performance is obviously improved. In addition, hand tracking is conducted after gesture recognition, which prevents the escape of the hand from the camera view range, so it ensures that the hand used for palmvein recognition is identical to the hand used during gesture recognition. The experimental results show that the proposed gesture recognition method performs satisfactorily on our dataset, and the hand tracking method has good robustness.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Hand Gesture Recognition Systems · Scientific and Engineering Research Topics · Face recognition and analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Laboratory of Jiangxi Province for Image Processing and Pattern Recognition, Nanchang Hangkong University, Nanchang 330063, China
Department of AI Engineering, Sookmyung Women’s University, Seoul 04310, Republic of Korea
Funding
National Natural Science Foundation of China · Technology Innovation Guidance Program Project of Jiangxi Province (Special Project of Technology Cooperation), China
References (51)
Show source-provided references
1. Human-machine interaction sensing technology based on hand gesture recognition: A review10.1109/THMS.2021.3086003 ↗
2. Biometrics in the Era of COVID-19: Challenges and Opportunities10.1109/TTS.2022.3203571 ↗
3. Masi, I., Wu, Y., Hassner, T., and Natarajan, P. (November, January 29). Deep face recognition: A survey. Proceedings of the 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Parana, Brazil.10.1109/SIBGRAPI.2018.00067 ↗
4. Palmprint recognition based on complete direction representation10.1109/TIP.2017.2705424 ↗
5. Feature extraction methods for palmprint recognition: A survey and evaluation10.1109/TSMC.2018.2795609 ↗
6. Decade progress of palmprint recognition: A brief survey10.1016/j.neucom.2018.03.081 ↗
7. Alignment-free row-co-occurrence cancelable palmprint fuzzy vault10.1016/j.patcog.2015.01.021 ↗
8. Multi-scale and multi-direction GAN for CNN-based single palm-vein identification10.1109/TIFS.2021.3059340 ↗
9. Magadia, A.P.I.D., Zamora, R.F.G.L., Linsangan, N.B., and Angelia, H.L.P. (2020, January 3–7). Bimodal hand vein recognition system using support vector machine. Proceedings of the 2020 IEEE 12th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), Manila, Philippines.10.1109/HNICEM51456.2020.9400017 ↗
10. Palmprint-palmvein fusion recognition based on deep hashing network10.1109/ACCESS.2021.3112513 ↗
11. Label enhancement-based multiscale transformer for palm-vein recognition
12. An optimized elman neural network for contactless palm-vein recognition framework10.1007/s11277-023-10579-x ↗
13. Presentation attacks in palmprint recognition systems10.33851/JMIS.2022.9.2.103 ↗
14. Wang, F., Leng, L., Teoh, A.B.J., and Chu, J. (2020). Palmprint false acceptance attack with a generative adversarial network (GAN). Appl. Sci., 10.10.3390/app10238547 ↗
15. Hand gesture based remote control system using infrared sensors and a camera10.1109/TCE.2014.7027342 ↗
16. Gesture recognition using reflected visible and infrared lightwave signals10.1109/THMS.2020.3043302 ↗
17. García-Bautista, G., Trujillo-Romero, F., and Caballero-Morales, S.O. (2017, January 22–24). Mexican sign language recognition using kinect and data time warping algorithm. Proceedings of the 2017 International Conference on Electronics, Communications and Computers (CONIELECOMP), Cholula, Mexico.10.1109/CONIELECOMP.2017.7891832 ↗
18. Mantecón, T., Del-Blanco, C.R., Jaureguizar, F., and García, N. (2019). A real-time gesture recognition system using near-infrared imagery. PLoS ONE, 14.10.1371/journal.pone.0223320 ↗
19. A multimodal framework for sensor based sign language recognition10.1016/j.neucom.2016.08.132 ↗
20. Multi-task pre-training with soft biometrics for transfer-learning palmprint recognition10.1007/s11063-022-10822-9 ↗
21. Park, H.J., Kang, J.W., and Kim, B.G. (2023). ssFPN: Scale sequence (S 2) feature-based feature pyramid network for object detection. Sensors, 23.10.3390/s23094432 ↗
22. Mujahid, A., Awan, M.J., Yasin, A., Mohammed, M.A., Damaševičius, R., Maskeliūnas, R., and Abdulkareem, K.H. (2021). Real-time hand gesture recognition based on deep learning YOLOv3 model. Appl. Sci., 11.10.3390/app11094164 ↗
23. Multi-sensor guided hand gesture recognition for a teleoperated robot using a recurrent neural network10.1109/LRA.2021.3089999 ↗
24. Handwriting recognition using wasserstein metric in adversarial learning10.1007/s42979-022-01445-x ↗
25. Multi-order texture features for palmprint recognition10.1007/s10462-022-10194-5 ↗
26. Dual-source discrimination power analysis for multi-instance contactless palmprint recognition10.1007/s11042-015-3058-7 ↗
27. Palmhash code vs. palmphasor code10.1016/j.neucom.2012.08.028 ↗
28. Sahoo, J.P., Prakash, A.J., Pławiak, P., and Samantray, S. (2022). Real-time hand gesture recognition using fine-tuned convolutional neural network. Sensors, 22.10.3390/s22030706 ↗
29. Vision based two-level hand tracking system for dynamic hand gestures in indoor environment10.1007/s11042-021-10669-7 ↗
30. Kulshreshth, A., Zorn, C., and LaViola, J.J. (2013, January 16–17). Poster: Real-time markerless kinect based finger tracking and hand gesture recognition for HCI. Proceedings of the 2013 IEEE Symposium on 3D User Interfaces (3DUI), Orlando, FL, USA.10.1109/3DUI.2013.6550241 ↗
31. Houston, A., Walters, V., Corbett, T., and Coppack, R. (2021). Evaluation of a multi-sensor Leap Motion setup for biomechanical motion capture of the hand. J. Biomech., 127.10.1016/j.jbiomech.2021.110713 ↗
32. Novel adaptive sensor fusion methodology for hand pose estimation with multileap motion10.1109/TIM.2021.3063752 ↗
33. Nonnarit, O., Ratchatanantakit, N., Tangnimitchok, S., Ortega, F., and Barreto, A. (2019, January 23–27). Hand tracking interface for virtual reality interaction based on marg sensors. Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan.
34. MagIK: A hand-tracking magnetic positioning system based on a kinematic model of the hand10.1109/TIM.2021.3065761 ↗
35. Mueller, F., Bernard, F., Sotnychenko, O., Mehta, D., Sridhar, S., Casas, D., and Theobalt, C. (2018, January 18–22). Ganerated hands for real-time 3d hand tracking from monocular rgb. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00013 ↗
36. MEgATrack: Monochrome egocentric articulated hand-tracking for virtual reality10.1145/3386569.3392452 ↗
37. Efficient deep learning: A survey on making deep learning models smaller, faster, and better10.1145/3578938 ↗
38. Machine learning and deep learning: A review of methods and applications
39. Zhuang, Q., Gan, S., and Zhang, L. (2022). Human-computer interaction based health diagnostics using ResNet34 for tongue image classification. Comput. Methods Programs Biomed., 226.10.1016/j.cmpb.2022.107096 ↗
40. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
41. Hu, J., Shen, L., and Sun, G. (2018, January 18–22). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
42. An improved neural network based on SENet for sleep stage classification10.1109/JBHI.2022.3157262 ↗
43. Zhang, F., Bazarevsky, V., Vakunov, A., Tkachenka, A., Sung, G., Chang, C.L., and Grundmann, M. (2020). Mediapipe hands: On-device real-time hand tracking. arXiv.
44. Ghanbari, S., Ashtyani, Z.P., and Masouleh, M.T. (2022, January 17–19). User identification based on hand geometrical biometrics using media-pipe. Proceedings of the 2022 30th International Conference on Electrical Engineering (ICEE), Seoul, Korea.10.1109/ICEE55646.2022.9827056 ↗
45. Güney, G., Jansen, T.S., Dill, S., Schulz, J.B., Dafotakis, M., Hoog Antink, C., and Braczynski, A.K. (2022). Video-based hand movement analysis of parkinson patients before and after medication using high-frame-rate videos and MediaPipe. Sensors, 22.10.3390/s22207992 ↗
46. Peiming, G., Shiwei, L., Liyin, S., Xiyu, H., Zhiyuan, Z., Mingzhe, C., and Zhenzhen, L. (2020, January 14–16). A PyQt5-based GUI for operational verification of wave forcasting system. Proceedings of the 2020 International Conference on Information Science, Parallel and Distributed Systems (ISPDS), Xi’an, China.10.1109/ISPDS51347.2020.00049 ↗
47. Renyi, L., and Yingzi, T. (2021, January 29–30). Semi-automatic marking system for robot competition based on PyQT5. Proceedings of the 2021 International Conference on Intelligent Computing, Automation and Systems (ICICAS), Chongqing, China.10.1109/ICICAS53977.2021.00058 ↗
48. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
49. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
50. Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022, January 18–24). A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01167 ↗
51. Chen, J., Kao, S.h., He, H., Zhuo, W., Wen, S., Lee, C.H., and Chan, S.H.G. (2023, January 18–22). Run, don’t walk: Chasing higher FLOPS for faster neural networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01157 ↗