A New Partitioned Spatial–Temporal Graph Attention Convolution Network for Human Motion Recognition
Source abstract
Abstract
At present, human action recognition can be used in all walks of life, because the skeleton can transmit intuitive information without being affected by environmental factors. However, it only focuses on local information. In order to solve these problems, we introduce a neural network model for human body recognition in this paper. We propose a model named NEW-STGCN-CA. The model is based on a spatial–temporal graph convolution network (ST-GCN), which contains a new partition strategy and coordination attention (CA) mechanism. By integrating the CA attention mechanism model, we enable the network to focus on input-related information, ignore unnecessary information, and prevent information loss. Second, a new partitioning strategy is proposed for the sampled regions, which is used to enhance the connection between local information and global information. We proved that the Top-1 accuracy of the NEW-STGCN-CA model in the NTU-RGB+D 60 dataset reached 84.86%, which was 1.7% higher than the original model; the accuracy of Top-1 on the Kinetics-Skeleton dataset reached 32.40%, which was 3.17% higher than the original model. The experimental results show that NEW-STGCN-CA can effectively improve the algorithm’s accuracy while also having high robustness and performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Pose and Action Recognition · Gait Recognition and Analysis · Context-Aware Activity Recognition Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China
References (41)
Show source-provided references
1. A Survey on Visual Surveillance of Object Motion and Behaviors10.1109/TSMCC.2004.829274 ↗
2. Ravanbakhsh, M., Mousavi, H., Rastegari, M., Murino, V., and Davis, L.S. (2015). Action Recognition with Image Based CNN Features. arXiv.
3. Liu, C., Fu, R., Li, Y., Gao, Y., Shi, L., and Li, W. (2021). A Self-Attention Augmented Graph Convolutional Clustering Networks for Skeleton-Based Video Anomaly Behavior Detection. Appl. Sci., 12.10.3390/app12010004 ↗
4. Zhu, Q., Deng, H., and Wang, K. (2022). Skeleton Action Recognition Based on Temporal Gated Unit and Adaptive Graph Convolution. Electronics, 11.10.3390/electronics11182973 ↗
5. Yang, S., Li, Q., He, D., Wang, J., and Li, D. (2022). Global Correlation Enhanced Hand Action Recognition Based on NST-GCN. Electronics, 11.10.3390/electronics11162518 ↗
6. Spatio-Temporal Attention-Based LSTM Networks for 3D Action Recognition and Detection10.1109/TIP.2018.2818328 ↗
7. Yan, S., Xiong, Y., and Lin, D. (2018, January 2–7). Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12328 ↗
8. Long, J., Shelhamer, E., and Darrell, T. (2017, January 21–26). Fully Convolutional Networks for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.
9. Hou, Q., Zhou, D., and Feng, J. (2021, January 20–25). Coordinate Attention for Efficient Mobile Network Design. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01350 ↗
10. Monoclonal antibodies against surface antigens of lymphoblasts and blood cells or bone marrow recognize constituents of the human nephron
11. Kim, T.S., and Reiter, A. (2017, January 21–26). Interpretable 3D Human Action Analysis with Temporal Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA.10.1109/CVPRW.2017.207 ↗
12. Bo, L., Dai, Y., Cheng, X., Chen, H., and He, M. (2017, January 10–14). Skeleton based action recognitionSkeleton based action recognition using translation-scale invariant image mapping and multi-scale deep CNN. Proceedings of the 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Hong Kong, China.10.1109/ICMEW.2017.8026282 ↗
13. Multiview-Based 3-D Action Recognition Using Deep Networks10.1109/THMS.2018.2883001 ↗
14. Yang, F., Wu, Y., Sakti, S., and Nakamura, S. (2019, January 16–18). Make Skeleton-based Action Recognition Model Smaller, Faster and Better. Proceedings of the ACM Multimedia Asia 2019, Beijing, China.10.1145/3338533.3366569 ↗
15. Ke, Q., Bennamoun, M., An, S., Sohel, F., and Boussaid, F. (2017, January 21–26). A New Representation of Skeleton Sequences for 3D Action Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.486 ↗
16. Skeleton-Based Action Recognition With Gated Convolutional Neural Networks10.1109/TCSVT.2018.2879913 ↗
17. Song, S., Lan, C., Xing, J., Zeng, W., and Liu, J. (2016, January 12–17). An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v31i1.11212 ↗
18. Du, Y., Wang, W., and Wang, L. (2015, January 7–12). Hierarchical recurrent neural network for skeleton based action recognition. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.
19. Zhang, P., Lan, C., Xing, J., Zeng, W., Xue, J., and Zheng, N. (2017, January 22–29). View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.233 ↗
20. Li, S., Li, W., Cook, C., Zhu, C., and Gao, Y. (2018, January 18–22). Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018, Salt Lake City, UT, USA.10.1109/CVPR.2018.00572 ↗
21. On the information and representation of non-Euclidean pairwise data10.1016/j.patcog.2006.04.016 ↗
22. Vision-based concrete crack detection using a hybrid framework considering noise effect10.1016/j.jobe.2022.105246 ↗
23. Shahroudy, A., Liu, J., Ng, T.T., and Wang, G. (2016, January 27–30). NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.115 ↗
24. Torsional capacity evaluation of RC beams using an improved bird swarm algorithm optimised 2D convolutional neural network10.1016/j.engstruct.2022.115066 ↗
25. Velikovi, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. (2017). Graph Attention Networks. arXiv.
26. Two-Stream Convolutional Networks for Action Recognition in Videos
27. Xu, M., Zhao, C., Rojas, D.S., Thabet, A., and Ghanem, B. G-TAD: Sub-Graph Localization for Temporal Action Detection, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020, Seattle, WA, USA, 13–19 June 2020.10.1109/CVPR42600.2020.01017 ↗
28. Zhang, X., Xu, C., and Tao, D. (2020, January 13–19). Context Aware Graph Convolution for Skeleton-Based Action Recognition. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01434 ↗
29. Lee, J., Lee, I., and Kang, J. (2019, January 10–15). Self-Attention Graph Pooling. Proceedings of the International Conference on Machine Learning 2019, Long Beach, CA, USA.
30. A Weakly Supervised Graph Deep Learning Framework for Point Cloud Registration
31. Pay Attention to Evolution: Time Series Forecasting with Deep Graph-Evolution Learning10.1109/TPAMI.2021.3076155 ↗
32. Zhang, J., Xie, W., Wang, C., Tu, R., and Tu, Z. (2022). Graph-aware transformer for skeleton-based action recognition. Vis. Comput., 1–12.10.1007/s00371-022-02603-1 ↗
33. Chen, C.H., and Ramanan, D. (2017, January 21–26). 3D Human Pose Estimation = 2D Pose Estimation + Matching. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.610 ↗
34. Zhe, C., Simon, T., Wei, S.E., and Sheikh, Y. (2017, January 21–26). Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.
35. Two-Stream Collaborative Learning with Spatial-Temporal Attention for Video Classification10.1109/TCSVT.2018.2808685 ↗
36. Human skeleton tracking from depth data using geodesic distances and optical flow
37. Jie, H., Li, S., Gang, S., and Albanie, S. (2017, January 21–26). Squeeze-and-Excitation Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.
38. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018). CBAM: Convolutional Block Attention Module, Springer.10.1007/978-3-030-01234-2_1 ↗
39. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. (2018, January 18–23). MobileNetV2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
40. He, X., Cheng, R., Zheng, Z., and Wang, Z. (2021). Small Object Detection in Traffic Scenes Based on YOLO-MXANet. Sensors, 21.10.3390/s21217422 ↗
41. Tan, M., and Le, Q.V. (2019, January 10–15). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the International Conference on Machine Learning 2019, Long Beach, CA, USA.