Dynamic Graph Attention Network for Skeleton-Based Action Recognition
Source abstract
Abstract
Skeleton-based human action recognition has garnered significant attention for its robustness to background noise and illumination variations. However, existing methods relying on Graph Convolutional Networks (GCNs) and Transformers exhibit inherent limitations: GCNs struggle to model interactions between non-adjacent joints due to predefined skeletal topology, while Transformers accumulate noise through unrestricted global dependency modeling. To address these challenges, we propose a Dynamic Graph Attention Network (DGAN) that dynamically integrates local structural features and global spatiotemporal dependencies. DGAN employs a masked attention mechanism to adaptively adjust node connectivity, forming a dynamic adjacency matrix that extends beyond physical skeletal constraints by selectively incorporating highly correlated joints. Additionally, a node-partition bias strategy is introduced to prioritize attention on collaboratively moving body parts, thereby enhancing discriminative feature extraction. Extensive experiments on the NTU RGB+D 60 and NTU RGB+D 120 datasets validate the effectiveness of DGAN, which outperforms state-of-the-art methods by achieving a balance between local topology preservation and global interaction modeling. Our approach provides a robust framework for skeleton-driven action recognition, demonstrating superior generalization across diverse scenarios.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Pose and Action Recognition · Anomaly Detection Techniques and Applications · Gait Recognition and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information and Control Engineering, China University of Mining and Technology, Xuzhou 221116, China
School of Information and Electrical Engineering, Xuzhou University of Technology, Xuzhou 221018, China
Funding
Xuzhou Science and Technology Plan Project
References (46)
Show source-provided references
1. Human action recognition from various data modalities: A review
2. Ren, B., Liu, M., Ding, R., and Liu, H. (2024). A survey on 3d skeleton-based action recognition using learning method. Cyborg Bionic Syst., 5.10.34133/cbsystems.0100 ↗
3. Bertasius, G., Wang, H., and Torresani, L. (2021, January 18–24). Is space-time attention all you need for video understanding?. Proceedings of the ICML, Online.
4. Liu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., and Hu, H. (2022, January 18–24). Video swin transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00320 ↗
5. Nguyen, H.P., and Ribeiro, B. (2023). Video action recognition collaborative learning with dynamics via PSO-ConvNet Transformer. Sci. Rep., 13.10.1038/s41598-023-39744-9 ↗
6. Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lučić, M., and Schmid, C. (2021, January 10–17). Vivit: A video vision transformer. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00676 ↗
7. Li, C., Zhong, Q., Xie, D., and Pu, S. (2017, January 10–14). Skeleton-based action recognition with convolutional neural networks. Proceedings of the 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Hong Kong.
8. Liu, J., Shahroudy, A., Xu, D., and Wang, G. (2016, January 11–14). Spatio-temporal lstm with trust gates for 3d human action recognition. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46487-9_50 ↗
9. Song, S., Lan, C., Xing, J., Zeng, W., and Liu, J. (2017, January 4–9). An end-to-end spatio-temporal attention model for human action recognition from skeleton data. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.11212 ↗
10. Liu, H., Tu, J., and Liu, M. (2017). Two-stream 3d convolutional neural network for skeleton-based action recognition. arXiv.
11. Li, B., Dai, Y., Cheng, X., Chen, H., Lin, Y., and He, M. (2017, January 10–14). Skeleton based action recognition using translation-scale invariant image mapping and multi-scale deep CNN. Proceedings of the 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Hong Kong.
12. Skeleton-based human action recognition with global context-aware attention LSTM networks10.1109/TIP.2017.2785279 ↗
13. Huang, L., Huang, Y., Ouyang, W., and Wang, L. (2020, January 7–12). Part-level graph convolutional network for skeleton-based action recognition. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i07.6759 ↗
14. Chi, H.-G., Ha, M.H., Chi, S., Lee, S.W., Huang, Q., and Ramani, K. (2022, January 18–24). Infogcn: Representation learning for human skeleton-based action recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01955 ↗
15. Zhou, Y., Cheng, Z.Q., Li, C., Fang, Y., Geng, Y., Xie, X., and Keuper, M. (2022). Hypergraph transformer for skeleton-based action recognition. arXiv.
16. Shahroudy, A., Liu, J., Ng, T.T., and Wang, G. (2016, January 27–30). Ntu rgb+ d: A large scale dataset for 3d human activity analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.115 ↗
17. Park, K., Patten, T., and Vincze, M. (November, January 27). Pix2pose: Pixel-wise coordinate regression of objects for 6d pose estimation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
18. Si, C., Chen, W., Wang, W., Wang, L., and Tan, T. (2019, January 15–20). An attention enhanced graph convolutional lstm network for skeleton-based action recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00132 ↗
19. Zhu, W., Lan, C., Xing, J., Zeng, W., Li, Y., Shen, L., and Xie, X. (2016, January 12–17). Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10451 ↗
20. Ding, Z., Wang, P., Ogunbona, P.O., and Li, W. (2017, January 10–14). Investigation of different skeleton features for cnn-based 3d action recognition. Proceedings of the 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Hong Kong.
21. Yan, S., Xiong, Y., and Lin, D. (2018, January 2–7). Spatial temporal graph convolutional networks for skeleton-based action recognition. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LO, USA.10.1609/aaai.v32i1.12328 ↗
22. Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (2018). Graph attention networks. arXiv.
23. Hu, L., Liu, S., and Feng, W. (2022). Spatial temporal graph attention network for skeleton-based action recognition. arXiv.
24. Spatial–temporal dynamic graph attention network for skeleton-based action recognition10.1109/ACCESS.2023.3247820 ↗
25. Plizzari, C., Cannici, M., and Matteucci, M. (2021, January 10–15). Spatial temporal transformer network for skeleton-based action recognition. Proceedings of the Pattern Recognition. ICPR International Workshops and Challenges, Virtual Event. Proceedings, Part III.10.1007/978-3-030-68796-0_50 ↗
26. Transformer for skeleton-based action recognition: A review of recent advances10.1016/j.neucom.2023.03.001 ↗
27. Gao, Z., Wang, P., Lv, P., Jiang, X., Liu, Q., Wang, P., Xu, M., and Li, W. (2022, January 4–8). Focal and global spatial-temporal transformer for skeleton-based action recognition. Proceedings of the Asian Conference on Computer Vision, Macao, China.10.1007/978-3-031-26316-3_10 ↗
28. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. arXiv.
29. Kipf, T.N., and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv.
30. Ye, F., Pu, S., Zhong, Q., Li, C., Xie, D., and Tang, H. (2020, January 10–16). Dynamic gcn: Context-enriched topology learning for skeleton-based action recognition. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.10.1145/3394171.3413941 ↗
31. Lu, C., Chen, H., Li, M., and Jing, L. (2024). Attention-guided and topology-enhanced shift graph convolutional network for skeleton-based action recognition. Electronics, 13.10.3390/electronics13183737 ↗
32. Chen, Y., Zhang, Z., Yuan, C., Li, B., Deng, Y., and Hu, W. (2021, January 11–17). Channel-wise topology refinement graph convolution for skeleton-based action recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.01311 ↗
33. Joint-Partition Group Attention for skeleton-based action recognition10.1016/j.sigpro.2024.109592 ↗
34. Zhang, P., Lan, C., Xing, J., Zeng, W., Xue, J., and Zheng, N. (2017, January 22–29). View adaptive recurrent neural networks for high performance human action recognition from skeleton data. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.233 ↗
35. Li, S., Li, W., Cook, C., Zhu, C., and Gao, Y. (2018, January 18–23). Independently recurrent neural network (indrnn): Building a longer and deeper rnn. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00572 ↗
36. Soo Kim, T., and Reiter, A. (2017, January 21–26). Interpretable 3d human action analysis with temporal convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.207 ↗
37. Li, C., Zhong, Q., Xie, D., and Pu, S. (2018). Co-occurrence feature learning from skeleton data for action recognition and detection with hierarchical aggregation. arXiv.10.24963/ijcai.2018/109 ↗
38. Xu, K., Ye, F., Zhong, Q., and Xie, D. (March, January 22). Topology-aware convolutional neural network for efficient skeleton-based action recognition. Proceedings of the AAAI Conference on Artificial Intelligence, Online.
39. Li, M., Chen, S., Chen, X., Zhang, Y., Wang, Y., and Tian, Q. (2019, January 15–20). Actional-structural graph convolutional networks for skeleton-based action recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00371 ↗
40. Shi, L., Zhang, Y., Cheng, J., and Lu, H. (2019, January 15–20). Two-stream adaptive graph convolutional networks for skeleton-based action recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.01230 ↗
41. Shi, L., Zhang, Y., Cheng, J., and Lu, H. (2019, January 15–20). Skeleton-based action recognition with directed graph neural networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00810 ↗
42. Cheng, K., Zhang, Y., He, X., Chen, W., Cheng, J., and Lu, H. (2020, January 13–19). Skeleton-based action recognition with shift graph convolutional network. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00026 ↗
43. H2GCN: A hybrid hypergraph convolution network for skeleton-based action recognition10.1016/j.jksuci.2024.102072 ↗
44. Cho, S., Maqbool, M., Liu, F., and Foroosh, H. (2020, January 1–5). Self-attention network for skeleton-based human action recognition. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA.10.1109/WACV45572.2020.9093639 ↗
45. Shi, L., Zhang, Y., Cheng, J., and Lu, H. (December, January 30). Decoupled spatial-temporal attention network for skeleton-based action-gesture recognition. Proceedings of the Asian Conference on Computer Vision, Kyoto, Japan.
46. Spatio-temporal segments attention for skeleton-based action recognition10.1016/j.neucom.2022.10.084 ↗