Video-Based Recognition of Human Activity Using Novel Feature Extraction Techniques
Source abstract
Abstract
This paper proposes a novel approach to activity recognition where videos are compressed using video coding to generate feature vectors based on compression variables. We propose to eliminate the temporal domain of feature vectors by computing the mean and standard deviation of each variable across all video frames. Thus, each video is represented by a single feature vector of 67 variables. As for the motion vectors, we eliminated their temporal domain by projecting their phases using PCA, thus representing each video by a single feature vector with a length equal to the number of frames in a video. Consequently, complex classifiers such as LSTM can be avoided and classical machine learning techniques can be used instead. Experimental results on the JHMDB dataset resulted in average classification accuracies of 68.8% and 74.2% when using the projected phases of motion vectors and video coding feature variables, respectively. The advantage of the proposed solution is the use of FVs with low dimensionality and simple machine learning techniques.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Pose and Action Recognition · Anomaly Detection Techniques and Applications · Video Analysis and Summarization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Engineering, American University of Sharjah, Sharjah P.O. Box 26666, United Arab Emirates
Funding
Open Access Program from the American University of Sharjah
References (28)
Show source-provided references
1. Wang, H., Klaser, A., Schmid, C., and Liu, C.-L. (2011, January 20–25). Action Recognition by Dense Trajectories. Proceedings of the CVPR 2011, Colorado Springs, CO, USA.10.1109/CVPR.2011.5995407 ↗
2. Learning Actionlet Ensemble for 3D Human Action Recognition10.1109/TPAMI.2013.198 ↗
3. Laptev, I., Marszalek, M., Schmid, C., and Rozenfeld, B. (2008, January 23–28). Learning Realistic Human Actions from Movies. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.10.1109/CVPR.2008.4587756 ↗
4. A Comprehensive Review of Recent Deep Learning Techniques for Human Activity Recognition10.1155/2022/8323962 ↗
5. Bilen, H., Fernando, B., Gavves, E., Vedaldi, A., and Gould, S. (2016, January 27–30). Dynamic Image Networks for Action Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.331 ↗
6. Ahsan, U., Madhok, R., and Essa, I. (2019, January 7–11). Video Jigsaw: Unsupervised Learning of Spatiotemporal Context for Video Action Recognition. Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa Village, HI, USA.10.1109/WACV.2019.00025 ↗
7. Ghadiyaram, D., Tran, D., and Mahajan, D. (2019, January 16–20). Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01232 ↗
8. Li, C., Zhong, Q., Xie, D., and Pu, S. (2019, January 16–20). Collaborative Spatiotemporal Feature Learning for Video Action Recognition. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00806 ↗
9. Khan, M.A., Javed, K., Khan, S.A., Saba, T., Habib, U., Khan, J.A., and Abbasi, A.A. (2020). Human Action Recognition Using Fusion of Multiview and Deep Features: An Application to Video Surveillance. Multimed. Tools Appl.10.1007/s11042-020-08806-9 ↗
10. Action Recognition in Videos Using Pre-Trained 2D Convolutional Neural Networks10.1109/ACCESS.2020.2983427 ↗
11. Stroud, J.C., Ross, D.A., Sun, C., Deng, J., and Sukthankar, R. (2020, January 1–5). D3D: Distilled 3D Networks for Video Action Recognition. Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA.10.1109/WACV45572.2020.9093274 ↗
12. Human Action Recognition Using Attention Based LSTM Network with Dilated CNN Features10.1016/j.future.2021.06.045 ↗
13. A Fast Human Action Recognition Network Based on Spatio-Temporal Features10.1016/j.neucom.2020.04.150 ↗
14. Learning Representative Temporal Features for Action Recognition10.1007/s11042-021-11022-8 ↗
15. Overview of the High Efficiency Video Coding (HEVC) Standard10.1109/TCSVT.2012.2221191 ↗
16. Random Forests10.1023/A:1010933404324 ↗
17. CNN and HEVC Video Coding Features for Static Video Summarization10.1109/ACCESS.2022.3188638 ↗
18. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
19. Jhuang, H., Gall, J., Zuffi, S., Schmid, C., and Black, M.J. (2013, January 1–8). Towards Understanding Action Recognition. Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.396 ↗
20. Choutas, V., Weinzaepfel, P., Revaud, J., and Schmid, C. (2018, January 18–22). PoTion: Pose MoTion Representation for Action Recognition. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00734 ↗
21. Chéron, G., Laptev, I., and Schmid, C. (2015). P-CNN: Pose-Based CNN Features for Action Recognition. arXiv.10.1109/ICCV.2015.368 ↗
22. Gkioxari, G., and Malik, J. (2014). Finding Action Tubes. arXiv.10.1109/CVPR.2015.7298676 ↗
23. Zhang, P., Xue, J., Lan, C., Zeng, W., Gao, Z., and Zheng, N. (2018). Adding Attentiveness to the Neurons in Recurrent Neural Networks. arXiv.10.1007/978-3-030-01240-3_9 ↗
24. Yan, A., Wang, Y., Li, Z., and Qiao, Y. (2019, January 16–20). PA3D: Pose-Action 3D Machine for Video Recognition. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00811 ↗
25. Multi-Region Two-Stream R-CNN for Action Detection10.1007/978-3-319-46493-0_45 ↗
26. Sun, C., Shrivastava, A., Vondrick, C., Sukthankar, R., Murphy, K., and Schmid, C. (2019). Relational Action Forecasting. arXiv.10.1109/CVPR.2019.00036 ↗
27. Cherian, A., Fernando, B., Harandi, M., and Gould, S. (2017). Generalized Rank Pooling for Activity Recognition. arXiv.10.1109/CVPR.2017.172 ↗
28. Bertasius, G., and Torresani, L. (2020, January 13–19). Classifying, segmenting, and tracking object instances in video with mask propagation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00976 ↗