Auxiliary Task Graph Convolution Network: A Skeleton-Based Action Recognition for Practical Use
Source abstract
Abstract
Graph convolution networks (GCNs) have been extensively researched for action recognition by estimating human skeletons from video clips. However, their image sampling methods are not practical because they require video-length information for sampling images. In this study, we propose an Auxiliary Task Graph Convolution Network (AT-GCN) with low and high-frame pathways while supporting a new sampling method. AT-GCN learns actions at a defined frame rate in the defined range with three losses: fuse, slow, and fast losses. AT-GCN handles the slow and fast losses in two auxiliary tasks, while the mainstream handles the fuse loss. AT-GCN outperforms the original State-of-the-Art model on the NTU RGB+D, NTU RGB+D 120, and NW-UCLA datasets while maintaining the same inference time. AT-GCN shows the best performance on the NTU RGB+D dataset at 90.3% from subjects, 95.2 from view benchmarks, on the NTU RGB+D 120 dataset at 86.5% from subjects, 87.6% from set benchmarks, and at 93.5% on the NW-UCLA dataset as top-1 accuracy.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Pose and Action Recognition · Anomaly Detection Techniques and Applications · Gait Recognition and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of AI Convergence, Chonnam National University, Gwangju 61186, Republic of Korea
SafeMotion, Gwangju 61011, Republic of Korea
Funding
Institute for Information and Communications Technology Planning and Evaluation (IITP) · Ministry of SMEs and Startups
References (36)
Show source-provided references
1. Construction site safety monitoring and excavator activity analysis system10.1007/s41693-022-00077-0 ↗
2. Sun, H., and Chen, Y. (2022, January 2–4). Real-time elderly monitoring for senior safety by lightweight human action recognition. Proceedings of the 2022 IEEE 16th International Symposium on Medical Information and Communication Technology (ISMICT), Lincoln, NE, USA.10.1109/ISMICT56646.2022.9828343 ↗
3. Zhou, H., Jiang, F., and Lu, H. (2022). Student Dangerous Behavior Detection in School. arXiv.
4. Yan, S., Xiong, Y., and Lin, D. (2018, January 2–7). Spatial temporal graph convolutional networks for skeleton-based action recognition. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12328 ↗
5. Chen, Y., Zhang, Z., Yuan, C., Li, B., Deng, Y., and Hu, W. (2021, January 11–17). Channel-wise topology refinement graph convolution for skeleton-based action recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.01311 ↗
6. Skeleton-based human action recognition via large-kernel attention graph convolutional network10.1109/TVCG.2023.3247075 ↗
7. Constructing stronger and faster baselines for skeleton-based action recognition10.1109/TPAMI.2022.3157033 ↗
8. Liu, M., Meng, F., Chen, C., and Wu, S. (2023, January 7–14). Novel motion patterns matter for practical skeleton-based action recognition. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.
9. Huang, X., Zhou, H., Wang, J., Feng, H., Han, J., Ding, E., Wang, J., Wang, X., Liu, W., and Feng, B. (2023). Graph contrastive learning for skeleton-based action recognition. arXiv.
10. Graph diffusion convolutional network for skeleton based semantic recognition of two-person actions
11. Multi-scale adaptive graph convolution network for skeleton-based action recognition10.1109/ACCESS.2024.3359234 ↗
12. Skeleton-based action recognition through contrasting two-stream spatial-temporal networks10.1109/TMM.2023.3239751 ↗
13. Feichtenhofer, C., Fan, H., Malik, J., and He, K. (November, January 27). Slowfast networks for video recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
14. Jiang, J., Chen, B., Pan, J., Wang, X., Liu, D., Jiang, J., and Long, M. (2024). Forkmerge: Mitigating negative transfer in auxiliary-task learning. Adv. Neural Inf. Process. Syst., 36.
15. Cai, Y., Sui, X., and Gu, G. (2023). Multi-modal multi-task feature fusion for RGBT tracking. Inf. Fusion, 97.10.1016/j.inffus.2023.101816 ↗
16. Candito, M. (2024). Auxiliary tasks to boost biaffine semantic dependency parsing. arXiv.
17. Shahroudy, A., Liu, J., Ng, T.-T., and Wang, G. (2016, January 27–30). Ntu rgb+ d: A large scale dataset for 3d human activity analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.115 ↗
18. Ntu rgb+ d 120: A large-scale benchmark for 3d human activity understanding10.1109/TPAMI.2019.2916873 ↗
19. Wang, J., Nie, X., Xia, Y., Wu, Y., and Zhu, S.-C. (2014, January 23–28). Cross-view action modeling, learning and recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.339 ↗
20. Ren, Y., Lan, Z., Liu, L., and Yu, H. (2024). Emsin: Enhanced multi-stream interaction network for vehicle trajectory prediction. IEEE Trans. Fuzzy Syst., 1–15.10.1109/TFUZZ.2024.3360946 ↗
21. Mao, K., Zhu, J., Su, L., Cai, G., Li, Y., and Dong, Z. (2023, January 7–14). FinalMLP: An enhanced two-stream MLP model for CTR prediction. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i4.25577 ↗
22. Yan, S., Xiong, X., Arnab, A., Lu, Z., Zhang, M., Sun, C., and Schmid, C. (2022, January 18–24). Multiview transformers for video recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00333 ↗
23. Yang, T., Zhu, Y., Xie, Y., Zhang, A., Chen, C., and Li, M. (2023). Aim: Adapting image models for efficient video action recognition. arXiv.
24. Wu, W., Wang, X., Luo, H., Wang, J., Yang, Y., and Ouyang, W. (2023, January 17–24). Bidirectional cross-modal knowledge exploration for video recognition with pre-trained vision-language models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00640 ↗
25. Xing, Z., Dai, Q., Hu, H., Chen, J., Wu, Z., and Jiang, Y.-G. (2023, January 17–24). Svformer: Semi-supervised video transformer for action recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01804 ↗
26. Wu, W., Sun, Z., and Ouyang, W. (2023, January 7–14). Revisiting classifier: Transferring vision-language models for video recognition. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i3.25386 ↗
27. Wu, C.-Y., Li, Y., Mangalam, K., Fan, H., Xiong, B., Malik, J., and Feichtenhofer, C. (2022, January 18–24). Memvit: Memory-augmented multiscale vision transformer for efficient long-term video recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01322 ↗
28. Human skeleton feature optimizer and adaptive structure enhancement graph convolution network for action recognition10.1109/TCSVT.2022.3201186 ↗
29. Wei, C., and Deng, Z. (2024, January 14–19). A Novel Contrastive Diffusion Graph Convolutional Network for Few-Shot Skeleton-Based Action Recognition. Proceedings of the ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10447668 ↗
30. Joint-bone fusion graph convolutional network for semi-supervised skeleton action recognition10.1109/TMM.2022.3168137 ↗
31. Shi, L., Zhang, Y., Cheng, J., and Lu, H. (2022). Action recognition via pose-based graph convolutional networks with intermediate dense supervision. Pattern Recognit., 121.10.1016/j.patcog.2021.108170 ↗
32. Chi, H.-g., Ha, M.H., Chi, S., Lee, S.W., Huang, Q., and Ramani, K. (2022, January 18–24). Infogcn: Representation learning for human skeleton-based action recognition. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01955 ↗
33. Lee, J., Lee, M., Lee, D., and Lee, S. (2023, January 1–6). Hierarchically decomposed graph convolutional networks for skeleton-based action recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00958 ↗
34. Zhang, P., Lan, C., Zeng, W., Xing, J., Xue, J., and Zheng, N. (2020, January 14–19). Semantics-guided neural networks for efficient skeleton-based human action recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00119 ↗
35. Loshchilov, I., and Hutter, F. (2016). Sgdr: Stochastic gradient descent with warm restarts. arXiv.
36. Liu, Z., Zhang, H., Chen, Z., Wang, Z., and Ouyang, W. (2020, January 14–19). Disentangling and unifying graph convolutions for skeleton-based action recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00022 ↗