Action Recognition via Adaptive Semi-Supervised Feature Analysis
Source abstract
Abstract
This study presents a new semi-supervised action recognition method via adaptive feature analysis. We assume that action videos can be regarded as data points in embedding manifold subspace, and their matching problem can be quantified through a specific Grassmannian kernel function while integrating feature correlation exploration and data similarity measurement into a joint framework. By maximizing the intra-class compactness based on labeled data, our algorithm can learn multiple features and leverage unlabeled data to enhance recognition. We introduce the Grassmannian kernels and the Projected Barzilai–Borwein (PBB) method to train a subspace projection matrix as a classifier. Experiment results show our method has outperformed the compared approaches when a few labeled training samples are available.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Pose and Action Recognition · Gait Recognition and Analysis · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mathematics and Computing Science, Guangxi Colleges and Universities Key Laboratory of Data Analysis and Computation, Guilin University of Electronic Technology, Guilin 541004, China
Center for Applied Mathematics, Guangxi (GUET), Guilin 541004, China
Anview.ai, Guilin 541010, China
School of Computer Engineering, Jingchu University of Technology, Jingmen 448000, China
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan 430072, China
Funding
National Natural Science Foundation of China · Science and Technology Project of Guangxi · Guangxi Key Laboratory of Automatic Detecting Technology and Instruments · Outstanding Youth Science and Technology Innovation Team Project of Colleges and Universities in Hubei Province · Key Science and Technology Project of Jingmen · Guangxi Key Research and Development Program
References (47)
Show source-provided references
1. Multi-level Feature Interaction and Efficient Non-Local Information Enhanced Channel Attention for image dehazing10.1016/j.neunet.2023.03.017 ↗
2. Scale-free heterogeneous cycleGAN for defogging from a single image for autonomous driving in fog10.1007/s00521-021-06296-w ↗
3. Precise Facial Landmark Detection by Reference Heatmap Transformer10.1109/TIP.2023.3261749 ↗
4. A Robust and Efficient Video Representation for Action Recognition10.1007/s11263-015-0846-5 ↗
5. Xu, Z., Hu, R., Chen, J., Chen, H., and Li, H. (2016, January 4–6). Global Contrast Based Salient Region Boundary Sampling for Action Recognition. Proceedings of the 22nd International Conference on MultiMedia Modeling, Miami, FL, USA.10.1007/978-3-319-27671-7_16 ↗
6. Singh, A., Chakraborty, O., Varshney, A., Panda, R., Feris, R., Saenko, K., and Das, A. (2021, January 20–25). Semi-supervised action recognition with temporal contrastive learning. Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01025 ↗
7. Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., and Van Gool, L. (2016, January 11–14). Temporal segment networks: Towards good practices for deep action recognition. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46484-8_2 ↗
8. Action recognition by saliency-based dense sampling10.1016/j.neucom.2016.09.106 ↗
9. Wang, S., Yang, Y., Ma, Z., Li, X., Pang, C., and Hauptmann, A.G. (2012, January 16–21). Action recognition by exploring data distribution and feature correlation. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.
10. Semi-supervised multiple feature analysis for action recognition10.1109/TMM.2013.2293060 ↗
11. An Adaptive Semisupervised Feature Analysis for Video Semantic Recognition10.1109/TCYB.2017.2647904 ↗
12. Semisupervised discriminant multimanifold analysis for action recognition10.1109/TNNLS.2018.2886008 ↗
13. Sultani, W., Chen, C., and Shah, M. (2018, January 18–23). Real-world anomaly detection in surveillance videos. Proceedings of the 2018 IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00678 ↗
14. Xu, Y., Yang, J., Cao, H., Mao, K., Yin, J., and See, S. (2021, January 8). Arid: A new dataset for recognizing action in the dark. Proceedings of the International Workshop on Deep Learning for Human Activity Recognition, Kyoto, Japan.10.1007/978-981-16-0575-8_6 ↗
15. Semisupervised Feature Analysis by Mining Correlations Among Multiple Tasks10.1109/TNNLS.2016.2582746 ↗
16. Sigurdsson, G.A., Russakovsky, O., and Gupta, A. (2017, January 22–29). What Actions are Needed for Understanding Human Actions in Videos?. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.235 ↗
17. Wang, X., Zhu, L., Wang, H., and Yang, Y. (2021, January 11–17). Interactive Prototype Learning for Egocentric Action Recognition. Proceedings of the 2021 IEEE International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00806 ↗
18. Ma, Y., Wang, Y., Wu, Y., Lyu, Z., Chen, S., Li, X., and Qiao, Y. (2022, January 10–14). Visual Knowledge Graph for Human Action Reasoning in Videos. Proceedings of the 30th ACM International Conference on Multimedia, Lisbon, Portugal.10.1145/3503161.3548257 ↗
19. Aktı, Ş., Tataroğlu, G.A., and Ekenel, H.K. (2019, January 6–9). Vision-based fight detection from surveillance cameras. Proceedings of the 2019 Ninth International Conference on Image Processing Theory, Tools and Applications, Istanbul, Turkey.10.1109/IPTA.2019.8936070 ↗
20. Modified subspace Barzilai-Borwein gradient method for non-negative matrix factorization10.1007/s10589-012-9507-6 ↗
21. Two-point step size gradient methods10.1093/imanum/8.1.141 ↗
22. Kernel analysis on Grassmann manifolds for action recognition10.1016/j.patrec.2013.01.008 ↗
23. Xiao, J., Jing, L., Zhang, L., He, J., She, Q., Zhou, Z., Yuille, A., and Li, Y. (2022, January 19–20). Learning from Temporal Gradient for Semi-supervised Action Recognition. Proceedings of the 2022 IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00325 ↗
24. Xu, Y., Wei, F., Sun, X., Yang, C., Shen, Y., Dai, B., Zhou, B., and Lin, S. (2022, January 19–20). Cross-model pseudo-labeling for semi-supervised action recognition. Proceedings of the 2022 IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00297 ↗
25. Si, C., Nie, X., Wang, W., Wang, L., Tan, T., and Feng, J. (2020, January 23–28). Adversarial self-supervised learning for semi-supervised 3d action recognition. Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58571-6_3 ↗
26. Kumar, A., and Rawat, Y.S. (2022, January 19–20). End-to-end semi-supervised learning for video action detection. Proceedings of the 2022 IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01429 ↗
27. Multiple feature analysis for infrared small target detection10.1109/LGRS.2017.2711047 ↗
28. Deep multimodal feature analysis for action recognition in rgb+d videos10.1109/TPAMI.2017.2691321 ↗
29. Stability of feature selection algorithm: A review
30. Image representation of pose-transition feature for 3D skeleton-based action recognition10.1016/j.ins.2019.10.047 ↗
31. Harandi, M.T., Sanderson, C., Shirazi, S., and Lovell, B.C. (2011, January 20–25). Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA.10.1109/CVPR.2011.5995564 ↗
32. Multitask linear discriminant analysis for view invariant action recognition10.1109/TIP.2014.2365699 ↗
33. CDMMA: Coupled discriminant multi-manifold analysis for matching low-resolution face images10.1016/j.sigpro.2015.09.026 ↗
34. Markovitz, A., Sharir, G., Friedman, I., Zelnik-Manor, L., and Avidan, S. (2020, January 13–19). Graph embedded pose clustering for anomaly detection. Proceedings of the 2020 IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01055 ↗
35. Dynamic graph convolutional networks10.1016/j.patcog.2019.107000 ↗
36. Cai, J., Fan, J., Guo, W., Wang, S., Zhang, Y., and Zhang, Z. (2022, January 19–20). Efficient deep embedded subspace clustering. Proceedings of the 2022 IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00012 ↗
37. Islam, A., and Radke, R. (2020, January 1–5). Weakly supervised temporal action localization using deep metric learning. Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision, Snowmass, CO, USA.10.1109/WACV45572.2020.9093620 ↗
38. A nearest-neighbor search model for distance metric learning10.1016/j.ins.2020.11.054 ↗
39. Human action recognition based on the Grassmann multi-graph embedding10.1007/s11760-018-1354-1 ↗
40. Action matching network: Open-set action recognition using spatio-temporal representation matching10.1007/s00371-019-01751-1 ↗
41. Spatial temporal graph deconvolutional network for skeleton-based human action recognition10.1109/LSP.2021.3049691 ↗
42. Multicategory Proximal Support Vector Machine Classifiers10.1007/s10994-005-0463-6 ↗
43. Web and Personal Image Annotation by Mining Label Correlation With Relaxed Visual Graph Embedding10.1109/TIP.2011.2169269 ↗
44. Jhuang, H., Gall, J., Zuffi, S., Schmid, C., and Black, M.J. (2013, January 1–8). Towards Understanding Action Recognition. Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.396 ↗
45. Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., and Serre, T. (2011, January 6–13). HMDB: A Large Video Database for Human Motion Recognition. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.10.1109/ICCV.2011.6126543 ↗
46. Soomro, K., Zamir, A.R., and Shah, M. (2012). UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild. arXiv.
47. Web Image Annotation Via Subspace-Sparsity Collaborated Feature Selection10.1109/TMM.2012.2187179 ↗