DualCascadeTSF-MobileNetV2: A Lightweight Violence Behavior Recognition Model
Source abstract
Abstract
This paper proposes a lightweight violent behavior recognition model, DualCascadeTSF-MobileNetV2, which is improved based on the temporal shift module and its subsequent research. By introducing the Dual Cascade Temporal Shift and Fusion module, the model further enhances the feature correlation ability in the time dimension and solves the problem of information sparsity caused by multiple temporal shifts. Meanwhile, the model incorporates the efficient lightweight structure of MobileNetV2, significantly reducing the number of parameters and computational complexity. Experiments were conducted on three public violent behavior datasets, Crowd Violence, RWF-2000, and Hockey Fights, to verify the performance of the model. The results show that it outperforms other classic models in terms of accuracy, computational speed, and memory size, especially among lightweight models. This research continues and expands on the previous achievements in the fields of TSM and lightweight network design, providing a new solution for real-time violent behavior recognition on edge devices.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Human Pose and Action Recognition · Network Security and Intrusion Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Laboratory of CTC & IE (Engineering University of PAP), Ministry of Education, Engineering University of PAP, Xi’an 710086, China
Graduate Student Brigade, Engineering University of PAP, Xi’an 710086, China
Funding
Science and Technology Innovation Project
References (40)
Show source-provided references
1. Lin, J., Gan, C., and Han, S. (November, January 27). TSM: Temporal Shift Module for Efficient Video Understanding. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00718 ↗
2. Lian, Z., Yin, Y., Lu, J., Xu, Q., Zhi, M., Hu, W., and Duan, W. (2024). A Survey: Feature Fusion Method for Object Detection Field. Advanced Intelligent Computing Technology and Applications, Springer Nature.10.1007/978-981-97-5588-2_8 ↗
3. Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., and Zhang, C. (2017, January 22–29). Learning Efficient Convolutional Networks through Network Slimming. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.298 ↗
4. Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, M.P. (2016). Pruning Filters for Efficient ConvNets. arXiv.
5. Sanh, V., Wolf, T., and Rush, A.M. (2020). Movement Pruning: Adaptive Sparsity by Fine-Tuning. arXiv.
6. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Wey, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv.
7. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. (2018, January 18–23). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
8. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18–23). ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00716 ↗
9. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size. arXiv.
10. Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., and Pang, R. (November, January 27). Searching for MobileNetV3. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00140 ↗
11. Tan, M., and Le, Q.V. (2019, January 10–15). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA.
12. Wang, X., Xiang, T., Zhang, C., Song, Y., Liu, D., Huang, H., and Cai, W. (2021). BiX-NAS: Searching Efficient Bi-directional Architecture for Medical Image Segmentation. Medical Image Computing and Computer Assisted Intervention–MICCAI 2021, Proceedings of the 24th International Conference, Strasbourg, France, 27 September–1 October 2021, Proceedings, Part I 24, Springer International Publishing.10.1007/978-3-030-87193-2_22 ↗
13. Yan, B., Peng, H., Wu, K., Wang, D., Fu, J., and Lu, H. (2021, January 20–25). LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01493 ↗
14. Cai, H., Gan, C., and Han, S. (2019). Once for All: Train One Network and Specialize it for Efficient Deployment. arXiv.
15. Violence behavior recognition of two-cascade temporal shift module with attention mechanism10.1117/1.JEI.30.4.043009 ↗
16. Gholami, A., Kwon, K., Wu, B., Tai, Z., Yue, X., Jin, P., Zhao, S., and Keutzer, K. (2018, January 18–22). SqueezeNext: Hardware-Aware Neural Network Design. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.10.1109/CVPRW.2018.00215 ↗
17. Wu, B., Wan, A., Yue, X., and Keutzer, K. (2018, January 18–22). Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00951 ↗
18. Qin, D., Leichner, C., Delakis, M., Fornoni, M., Luo, S., Yang, F., Wang, W., Banbury, C., Ye, C., and Akin, B. (2024). MobileNetV4: Universal Models for the Mobile Ecosystem. Computer Vision—ECCV 2024, Proceedings of the 18th European Conference, Milan, Italy, 29 September–4 October 2024, Proceedings, Part XL, Springer.10.1007/978-3-031-73661-2_5 ↗
19. Tan, M., and Le, Q.V. (2022, January 17–23). EfficientNetV2: Smaller Models and Faster Training. Proceedings of the 38th International Conference on Machine Learning, Baltimore, MD, USA.
20. Wang, A., Chen, H., Lin, Z., Han, J., and Ding, G. (2024, January 16–22). Rep ViT: Revisiting Mobile CNN From ViT Perspective. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.01506 ↗
21. Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., and Xu, C. (2020, January 14–19). GhostNet: More Features From Cheap Operations. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00165 ↗
22. Tang, Y., Han, K., and Guo, J. (December, January 28). GhostNetV2: Enhance Cheap Operation with Long-Range Attention. Proceedings of the Advances in Neural Information Processing Systems 35 (NeurIPS 2022), New Orleans, LA, USA.
23. GhostNets on Heterogeneous Devices via Cheap Operations10.1007/s11263-022-01575-y ↗
24. Ma, N., Zhang, X., Zheng, H.-T., and Sun, J. (2018, January 8–14). ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. Proceedings of the Computer Vision, 15th European Conference (ECCV 2018), Munich, Germany.10.1007/978-3-030-01264-9_8 ↗
25. Freeman, I., Roese-Koerner, L., and Kummert, A. (2018, January 7–10). EffNet: An Efficient Structure for Convolutional Neural Networks. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.10.1109/ICIP.2018.8451339 ↗
26. Hassner, T., Itcher, Y., and Kliper-Gross, O. (2012, January 16–21). Violent flows: Real-time detection of violent crowd behavior. Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA.10.1109/CVPRW.2012.6239348 ↗
27. Cheng, M., Cai, K., and Li, M. (2021, January 10–15). RWF-2000: An Open Large Scale Video Database for Violence Detection. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.10.1109/ICPR48806.2021.9412502 ↗
28. Bermejo Nievas, E., Deniz Suarez, O., Bueno Garcia, G., and Sukthankar, R. (2011, January 29–31). Violence Detection in Video Using Computer Vision Techniques. Proceedings of the International Conference on Computer Analysis of Images and Patterns, Seville, Spain.10.1007/978-3-642-23678-5_39 ↗
29. Zhang, Y., Li, Y., and Guo, S. (2022). Lightweight mobile network for real-time violence recognition. PLoS ONE, 17.10.1371/journal.pone.0276939 ↗
30. Shafiq, M., and Gu, Z. (2022). Deep Residual Learning for Image Recognition: A Survey. Appl. Sci., 12.10.3390/app12188972 ↗
31. 3D convolutional neural networks for human action recognition10.1109/TPAMI.2012.59 ↗
32. Long-Term Recurrent Convolutional Networks for Visual Recognition and Description10.1109/TPAMI.2016.2599174 ↗
33. Carreira, J., and Zisserman, A. (2017, January 22–25). Quo Vadis, action recognition? A new model and the kinetics dataset. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.502 ↗
34. Wang, W., Dong, S., Zou, K., and Li, W. (2022, January 7–9). A Lightweight Network for Violence Detection. Proceedings of the ICIGP 2022: 2022 the 5th International Conference on Image and Graphics Processing (ICIGP), Beijing, China.10.1145/3512388.3512391 ↗
35. Meng, Y., Lin, C.-C., Panda, R., Sattigeri, P., Karlinsky, L., Oliva, A., Saenko, K., and Feris, R. (2020). AR-Net: Adaptive Frame Resolution for Efficient Action Recognition. Computer Vision–ECCV 2020, Proceedings of the16th European Conference, Glasgow, UK, 23–28 August 2020, Proceedings, Part VII 16, Springer.10.1007/978-3-030-58571-6_6 ↗
36. Li, Y., Ji, B., Shi, X., Zhang, J., Kang, B., and Wang, L. (2020, January 14–19). TEA: Temporal Excitation and Aggregation for Action Recognition. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00099 ↗
37. Multi-Level Two-Stream Fusion-Based Spatio-Temporal Attention Model for Violence Detection and Localization10.1142/S0218001422550023 ↗
38. Video violence recognition and localization using a semi-supervised hard attention model10.1016/j.eswa.2022.118791 ↗
39. Not all temporal shift modules are profitable10.1117/1.JEI.31.4.043030 ↗
40. Temporal Segment Networks: Towards Good Practices for Deep Action Recognition