journal article·2022-01-17Open access

Human Activity Classification Using the 3DCNN Architecture

Roberta Vrskova, Robert Hudec, Patrik Kamencay, Peter Sykora

Applied Sciences· Volume 12· Issue 2

Abstract

Interest in utilizing neural networks in a variety of scientific and academic studies and in industrial applications is increasing. In addition to the growing interest in neural networks, there is also a rising interest in video classification. Object detection from an image is used as a tool for various applications and is the basis for video classification. Identifying objects in videos is more difficult than for single images, as the information in videos has a time continuity constraint. Common neural networks such as ConvLSTM (Convolutional Long Short-Term Memory) and 3DCNN (3D Convolutional Neural Network), as well as many others, have been used to detect objects from video. Here, we propose a 3DCNN for the detection of human activity from video data. The experimental results show that the optimized proposed 3DCNN provides better results than neural network architectures for motion, static and hybrid features. The proposed 3DCNN obtains the highest recognition precision of the methods considered, 87.4%. In contrast, the neural network architectures for motion, static and hybrid features achieve precisions of 65.4%, 63.1% and 71.2%, respectively. We also compare results with previous research. Previous 3DCNN architecture on database UCF Youtube Action worked worse than the architecture we proposed in this article, where the achieved result was 29%. The experimental results on the UCF YouTube Action dataset demonstrate the effectiveness of the proposed 3DCNN for recognition of human activity. For a more complex comparison of the proposed neural network, the modified UCF101 dataset, full UCF50 dataset and full UCF101 dataset were compared. An overall precision of 82.7% using modified UCF101 dataset was obtained. On the other hand, the precision using full UCF50 dataset and full UCF101 dataset was 80.6% and 78.5%, respectively.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Human Pose and Action Recognition · Anomaly Detection Techniques and Applications · Video Surveillance and Tracking Methods

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Multimedia and Information-Communication Technologies, University of Zilina, 010 26 Zilina, Slovakia

References (29)

Show source-provided references
1. Automatic handgun detection alarm in videos using deep learning10.1016/j.neucom.2017.05.012 ↗
2. Dhiman, C.H., and Vischakarma, D. (2017, January 14–16). High dimensional abnormal human activity recognition using histogram oriented gradients and Zernike moments. Proceedings of the International Conference on Computational Intelligence and Computing Research (ICCIC), Coimbatore, India.10.1109/ICCIC.2017.8524372 ↗
3. Peixoto, B., Avila, S., Dias, Z., and Rocha, A. (2018, January 27–30). Breaking down violence: A deep-learning strategy to model and classify violence in videos. Proceedings of the International Conference on Availability, Reliability and Security (ARES), Hamburg, Germany.10.1145/3230833.3232809 ↗
4. Review on state-of-the-art violence detection techniques10.1109/ACCESS.2019.2932114 ↗
5. Liu, J., Yang, Y., and Shah, M. (2009, January 20–25). Learning Semantic Visual Vocabularies using Diffusion Distance. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA.10.1109/CVPR.2009.5206845 ↗
6. Liu, J., Luo, J., and Shah, M. (2009, January 20–25). Recognizing Realistic Actions from Videos “in the Wild”. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA.10.1109/CVPR.2009.5206744 ↗
7. Zhang, X., Yao, L., Huang, C., Sheng, Q.Z., and Wang, X. (2017, January 14–18). Intent recognition in smart living through deep recurrent neural networks. Proceedings of the International Conference on Neural Information Processing (ICONIP), Guangzhou, China.10.1007/978-3-319-70096-0_76 ↗
8. Guo, Z.H., Chen, Y., Huang, W., Zhang, J.H., and Wang, X. (2019, January 17–19). An Efficient 3D-NAS Method for Video-Based Gesture Recognition. Proceedings of the International Conference on Artificial Neural Networks (ICANN), Munich, Germany.10.1007/978-3-030-30508-6_26 ↗
9. Hand sign language recognition using multi-view hand skeleton10.1016/j.eswa.2020.113336 ↗
10. An enhanced 3DCNN-ConvLSTM for spatiotemporal multimedia data analysis10.1002/cpe.5302 ↗
11. Castro-Vargas, J., Zapata-Impata, B., Gil, P., Garcia-Rodriguez, J., and Torres, F. (2019, January 19–21). 3DCNN Performance in Hand Gesture Recognition Applied to Robot Arm Interaction. Proceedings of the International Conference on Pattern Recognition Applications and Methods (ICPRAM), Prague, Czech Republic.10.5220/0007570208020806 ↗
12. A hybrid approach for search and rescue using 3DCNN and PSO10.1007/s00521-020-05001-7 ↗
13. Wang, Y.H., and Dantcheva, A. (2020, January 16–20). A video is worth more than 1000 lies. Comparing 3DCNN approaches for detecting deepfakes. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition and Workshops, Buenos Aires, Argentina.10.1109/FG47880.2020.00089 ↗
14. Hand Gesture Recognition for Sign Language Using 3DCNN
15. Temporal Fusion Approach for Video Classification with Convolutional and LSTM Neural Networks Applied to Violence Detection10.4114/intartif.vol24iss67pp40-50 ↗
16. Video action detection by learning graph-based spatio-temporal interactions10.1016/j.cviu.2021.103187 ↗
17. Mobile Neural Architecture Search Network and Convolutional Long Short-Term Memory-Based Deep Features Toward Detecting Violence from Video10.1007/s13369-021-05589-5 ↗
18. Learning to detect anomaly events in crowd scenes from synthetic data10.1016/j.neucom.2021.01.031 ↗
19. Action recognition using kinematics posture feature on 3D skeleton joint locations10.1016/j.patrec.2021.02.013 ↗
20. Human Action Recognition in Drone Videos using a Few Aerial Training Examples10.1016/j.cviu.2021.103186 ↗
21. Hou, H., Li, Y., Zhang, C., Liao, H., Zhang, Y., and Liu, Y. (2021, January 28–30). Vehicle Behavior Recognition using Multi-Stream 3D Convolutional Neural Network. Proceedings of the 2021 36th Youth Academic Annual Conference of Chinese Association of Automation (YAC), Nanchang, China.10.1109/YAC53711.2021.9486615 ↗
22. Vrskova, R., Hudec, R., Sykora, P., Kamencay, P., and Radilova, M. (2020, January 12–13). Education of Video Classification Based by Neural Networks. Proceedings of the International Conference on Emerging eLearning Technologies and Applications (ICETA), Košice, Slovakia.10.1109/ICETA51985.2020.9379190 ↗
23. Vrskova, R., Hudec, R., Sykora, P., Kamencay, P., and Benco, M. (2020, January 14–16). Violent Behavioral Activity Classification using Artificial Neural Network. Proceedings of the New Trends in Signal Processing (NTSP), Demanovska Dolina, Slovakia.10.1109/NTSP49686.2020.9229532 ↗
24. Deep learning serves voice cloning: How vulnerable are automatic speaker verification systems to spooting trial10.1109/MCOM.001.1900396 ↗
25. 3D Convolutional Neural Networks for Human Action Recognition10.1109/TPAMI.2012.59 ↗
26. Three-dimensional convolutional neural network (3D-CNN) for heterogeneous material homogenization10.1016/j.commatsci.2020.109850 ↗
27. Vrskova, R., Sykora, P., Kamencay, P., Hudec, R., and Radil, R. (2021, January 26–28). Hyperparameter Tuning of ConvLSTM Network Models. Proceedings of the 2021 44th International Conference on Telecommunications and Signal Processing (TSP), Brno, Czech Republic.10.1109/TSP52935.2021.9522683 ↗
28. Soomro, K., Zamir, A.R., and Shah, M. (2012). UCF101: A Dataset of 101 Human Action Classes From Videos in The Wild. arXiv.
29. Animal recognition system based on convolutional neural network