Multi-Person Action Recognition Based on Millimeter-Wave Radar Point Cloud
Source abstract
Abstract
Human action recognition has many application prospects in human-computer interactions, innovative furniture, healthcare, and other fields. The traditional human motion recognition methods have limitations in privacy protection, complex environments, and multi-person scenarios. Millimeter-wave radar has attracted attention due to its ultra-high resolution and all-weather operation. Many existing studies have discussed the application of millimeter-wave radar in single-person scenarios, but only some have addressed the problem of action recognition in multi-person scenarios. This paper uses a commercial millimeter-wave radar device for human action recognition in multi-person scenarios. In order to solve the problems of severe interference and complex target segmentation in multiplayer scenarios, we propose a filtering method based on millimeter-wave inter-frame differences to filter the collected human point cloud data. We then use the DBSCAN algorithm and the Hungarian algorithm to segment the target, and finally input the data into a neural network for classification. The classification accuracy of the system proposed in this paper reaches 92.2% in multi-person scenarios through experimental tests with the five actions we set.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Hand Gesture Recognition Systems · Gait Recognition and Analysis · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science and Engineering, Northwest Normal University, Lanzhou 730070, China
Gansu Province Internet of Things Engineering Research Center, Lanzhou 730070, China
Funding
National Natural Science Foundation of China · Industrial Support Foundations of Gansu
References (33)
Show source-provided references
1. Robust human activity recognition from depth video using spatiotemporal multi-fused features10.1016/j.patcog.2016.08.003 ↗
2. Physical human activity recognition using wearable sensors10.3390/s151229858 ↗
3. Chen, C., Jafari, R., and Kehtarnavaz, N. (2015, January 27–30). UTD-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor. Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada.10.1109/ICIP.2015.7350781 ↗
4. Multiple human tracking and identification with wireless distributed pyroelectric sensor systems10.1109/JSYST.2009.2035734 ↗
5. Han, J., and Bhanu, B. (2005, January 21–23). Human activity recognition in thermal infrared imagery. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)-Workshops, San Diego, CA, USA.
6. WiFi sensing with channel state information: A survey10.1145/3310194 ↗
7. Deep AI enabled ubiquitous wireless sensing: A survey10.1145/3379443 ↗
8. Wifall: Device-free fall detection by wireless networks10.1109/TMC.2016.2557792 ↗
9. Wang, W., Liu, A.X., Shahzad, M., Ling, K., and Lu, S. (2015, January 7–11). Understanding and modeling of wifi signal based human activity recognition. Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, Paris, France.10.1145/2789168.2790093 ↗
10. Iovescu, C., and Rao, S. (2017). The Fundamentals of Millimeter Wave Sensors, Texas Instruments.
11. Multistatic micro-Doppler radar for determining target orientation and activity classification10.1109/TAES.2015.130595 ↗
12. Singh, A.D., Sandha, S.S., Garcia, L., and Srivastava, M. (2019, January 25). Radhar: Human activity recognition from point clouds generated through a millimeter-wave radar. Proceedings of the 3rd ACM Workshop on Millimeter-Wave Networks and Sensing Systems, Los Cabos, Mexico.10.1145/3349624.3356768 ↗
13. Gong, P., Wang, C., and Zhang, L. (2021, January 18–22). Mmpoint-gnn: Graph neural network with dynamic edges for human activity recognition through a millimeter-wave radar. Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China.10.1109/IJCNN52387.2021.9533989 ↗
14. 3D convolutional neural networks for human action recognition10.1109/TPAMI.2012.59 ↗
15. Millimeter-wave wireless communications for IoT-cloud supported autonomous vehicles: Overview, design, and challenges10.1109/MCOM.2017.1600422CM ↗
16. An overview of signal processing techniques for millimeter wave MIMO systems10.1109/JSTSP.2016.2523924 ↗
17. DBSCAN revisited, revisited: Why and how you should (still) use DBSCAN10.1145/3068335 ↗
18. The Hungarian method for the assignment problem10.1002/nav.3800020109 ↗
19. Khan, M., Ahmad, J., El Saddik, A., Gueaieb, W., De Masi, G., and Karray, F. (2024, January 17–21). Drone-HAT: Hybrid Attention Transformer for Complex Action Recognition in Drone Surveillance Videos. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPRW63382.2024.00474 ↗
20. Mutegeki, R., and Han, D.S. (2020, January 19–21). A CNN-LSTM approach to human activity recognition. Proceedings of the 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Fukuoka, Japan.10.1109/ICAIIC48513.2020.9065078 ↗
21. A review of algorithms for filtering the 3D point cloud10.1016/j.image.2017.05.009 ↗
22. Pantomime: Mid-air gesture recognition with sparse millimeter-wave radar point clouds10.1145/3448110 ↗
23. Bevilacqua, A., MacDonald, K., Rangarej, A., Widjaya, V., Caulfield, B., and Kechadi, T. (2019). Human activity recognition with convolutional neural networks. Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10–14, 2018, Proceedings, Part III 18, Springer International Publishing.10.1007/978-3-030-10997-4_33 ↗
24. Pienaar, S.W., and Malekian, R. (2019, January 18–20). Human activity recognition using LSTM-RNN deep neural network architecture. Proceedings of the 2019 IEEE 2nd Wireless Africa Conference (WAC), Pretoria, South Africa.10.1109/AFRICA.2019.8843403 ↗
25. Duran, B.S., and Odell, P.L. (2013). Cluster Analysis: A Survey, Springer Science & Business Media.
26. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network10.1016/j.physd.2019.132306 ↗
27. Wang, F., and Tax, D.M. (2016). Survey on the attention based RNN model and its applications in computer vision. arXiv.
28. Abdu, F.J., Zhang, Y., Fu, M., Li, Y., and Deng, Z. (2021). Application of deep learning on millimeter-wave radar signals: A review. Sensors, 21.10.3390/s21061951 ↗
29. Texas Instruments (2023, October 23). AWR1843BOOST and IWR1843BOOST Single-Chip mmWave Sensing Solution User’s Guide (Rev. B). Available online: https://www.ti.com/tool/IWR1843BOOST.
30. Toward millimeter-wave joint radar communications: A signal processing perspective10.1109/MSP.2019.2913173 ↗
31. Passive millimeter wave imaging10.1109/MMW.2003.1237476 ↗
32. Zhang, Q., Wang, C., Wu, H., Xin, C., and Phuong, T.V. (2018, January 13–19). GELU-Net: A Globally Encrypted, Locally Unencrypted Deep Neural Network for Privacy-Preserved Learning. Proceedings of the International Joint Conference on Artificial Intelligence, Stockholm, Sweden.10.24963/ijcai.2018/547 ↗
33. LSTM-CNN architecture for human activity recognition10.1109/ACCESS.2020.2982225 ↗