HSTNet:Violent Action Detection
Source abstract
Abstract
To enhance public safety and safeguard lives and property, the automatic detection of anomalous and violent behaviors in video has become a key task in intelligent surveillance systems. Violent actions are often abrupt, rapid, and irregular, posing considerable challenges to conventional approaches. Existing methods based on hand-crafted features and convolutional neural networks still exhibit limitations in spatiotemporal feature extraction, recognition accuracy, and model robustness. To address these issues, this paper proposes HSTNet, a hybrid neural architecture that integrates Spiking Neural Networks (SNNs) with Transformers. The framework adopts a dual-branch design: the SNN branch models temporal dynamics in video, while the Transformer branch extracts spatial structural information. A feature interaction module is further introduced to enable deep cross-modal fusion. Experiments on multiple datasets including UCF101, HMDB51, Hockey Fight, and Movies Fight demonstrate that HSTNet achieves significantly higher accuracy than state-of-the-art baselines, indicating strong performance and promising application potential.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Pose and Action Recognition · Anomaly Detection Techniques and Applications · Emotion and Mood Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronic Information, Wuhan University, Wuhan 430072, China
References (52)
Show source-provided references
1. Naimi, S., Bouachir, W., Bilodeau, G.A., and Mishara, B. (2025, January 3–7). SSTAR: Skeleton-Based Spatio-Temporal Action Recognition for Intelligent Video Surveillance and Suicide Prevention in Metro Stations. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACVW65960.2025.00091 ↗
2. CNNs, RNNs and Transformers in Human Action Recognition: A Survey and a Hybrid Model10.1007/s10462-025-11388-3 ↗
3. A Review on the Use of Top-View Surveillance Videos for Pedestrian Detection, Tracking and Behavior Recognition across Public Spaces10.1016/j.aap.2025.107986 ↗
4. Application of Temporal Action Detection Technology in Abnormal Event Detection of Surveillance Video10.1109/ACCESS.2025.3533552 ↗
5. Research on Passengers’ Behavior Recognition Method in Public Transport Vehicles Based on Efficient 3D CNN10.1007/s00530-024-01612-5 ↗
6. Convolutional Attention-Based Bidirectional Recurrent Neural Network for Human Action Recognition10.1111/coin.70049 ↗
7. Two-stream spatio-temporal GCN-transformer networks for skeleton-based action recognition10.1038/s41598-025-87752-8 ↗
8. End-to-end temporal action detection with transformer10.1109/TIP.2022.3195321 ↗
9. Liu, X., Bai, S., and Bai, X. (2022, January 18–24). An empirical study of end-to-end temporal action detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01938 ↗
10. Mahbub, U., Imtiaz, H., and Ahad, M.A.R. (2011, January 22–24). An optical flow based approach for action recognition. Proceedings of the 14th International Conference on Computer and Information Technology (ICCIT 2011), Dhaka, Bangladesh.10.1109/ICCITechn.2011.6164868 ↗
11. Nam, J., Alghoniemy, M., and Tewfik, A.H. (1998, January 4–7). Audio-visual content-based violent scene characterization. Proceedings of the 1998 International Conference on Image Processing, Chicago, IL, USA.
12. Feichtenhofer, C., Pinz, A., and Zisserman, A. (2016, January 27–30). Convolutional two-stream network fusion for video action recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.213 ↗
13. Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., and Gool, L.V. (2016, January 11–14). Temporal segment networks: Towards good practices for deep action recognition. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46484-8_2 ↗
14. Ma, Y., Yuan, L., Abdelraouf, A., Han, K., Gupta, R., Li, Z., and Wang, Z. (2023, January 17–24). M2DAR: Multi-view multi-scale driver action recognition with vision transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00557 ↗
15. Ding, C., Fan, S., Zhu, M., Feng, W., and Jia, B. (2014). Violence detection in video using 3D convolutional neural networks. Advances in Visual Computing, ISVC 2014, Springer.10.1007/978-3-319-14364-4_53 ↗
16. Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., and Paluri, M. (2018, January 18–23). A closer look at spatiotemporal convolutions for action recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00675 ↗
17. Feichtenhofer, C. (2020, January 13–19). X3D: Expanding architectures for efficient video recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00028 ↗
18. Cheng, M., Cai, K., and Li, M. (2021, January 10–15). RWF-2000: An open large scale video database for violence detection. Proceedings of the 25th International Conference on Pattern Recognition, Milan, Italy.10.1109/ICPR48806.2021.9412502 ↗
19. Wang, T., Liu, H., Ding, R., Li, W., You, Y., and Li, X. (2023, January 4–10). Interweaved graph and attention network for 3D human pose estimation. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Rhodes Island, Greece.10.1109/ICASSP49357.2023.10097259 ↗
20. A comparative review of recent Kinect-based action recognition algorithms10.1109/TIP.2019.2925285 ↗
21. Wang, H., and Wang, L. (2017, January 21–26). Modeling temporal dynamics and spatial configurations of actions using two-stream recurrent neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.387 ↗
22. Ding, Z., Wang, P., Ogunbona, P.O., and Li, W. (2017, January 10–14). Investigation of different skeleton features for CNN-based 3D action recognition. Proceedings of the IEEE International Conference on Multimedia & Expo Workshops, Hong Kong, China.
23. Li, M., Chen, S., Chen, X., Zhang, Y., Wang, Y., and Tian, Q. (2019, January 15–20). Actional-structural graph convolutional networks for skeleton-based action recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00371 ↗
24. Cho, S., Maqbool, M., Liu, F., and Foroosh, H. (2020, January 1–5). Self-attention network for skeleton-based human action recognition. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA.10.1109/WACV45572.2020.9093639 ↗
25. Zhao, R., Ali, H., and Van der Smagt, P. (2017, January 24–28). Two-stream RNN/CNN for action recognition in 3D videos. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Vancouver, BC, Canada.10.1109/IROS.2017.8206288 ↗
26. Li, W., Wen, L., Chang, M.-C., Lim, S.N., and Lyu, S. (2017, January 22–29). Adaptive RNN tree for large-scale human action recognition. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.161 ↗
27. Ensemble one-dimensional convolution neural networks for skeleton-based action recognition10.1109/LSP.2018.2841649 ↗
28. Wang, P., Li, Z., Hou, Y., and Li, W. (2016, January 15–19). Action recognition based on joint trajectory maps using convolutional neural networks. Proceedings of the ACM International Conference on Multimedia, Amsterdam, The Netherlands.10.1145/2964284.2967191 ↗
29. Yan, S., Xiong, Y., and Lin, D. (2018, January 2–7). Spatial temporal graph convolutional networks for skeleton-based action recognition. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12328 ↗
30. Lee, J., Lee, M., Lee, D., and Lee, S. (2023, January 1–6). Hierarchically decomposed graph convolutional networks for skeleton-based action recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00958 ↗
31. Hypergraph neural network for skeleton-based action recognition10.1109/TIP.2021.3051495 ↗
32. Attention is all you need
33. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16×16 words: Transformers for image recognition at scale. arXiv.
34. Lee, Y., Kim, J., Willette, J., and Hwang, S.J. (2022, January 18–24). MPViT: Multi-path vision transformer for dense prediction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00714 ↗
35. Xia, Z., Pan, X., Song, S., Li, L.E., and Huang, G. (2022, January 18–24). Vision transformer with deformable attention. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00475 ↗
36. Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., and Dong, L. (2022, January 18–24). Swin Transformer V2: Scaling up capacity and resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01170 ↗
37. Ahn, D., Kim, S., Hong, H., and Ko, B.C. (2023, January 2–7). Star-transformer: A spatio-temporal cross attention transformer for human action recognition. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV56688.2023.00333 ↗
38. SEENN: Towards temporal spiking early exit neural networks
39. Self-supervised learning of event-based optical flow with spiking neural networks
40. Kosta, A.K., and Roy, K. (2023, January 11–13). Adaptive-SpikeNet: Event-based optical flow estimation using spiking neural networks with learnable neuronal dynamics. Proceedings of the IEEE International Conference on Robotics and Automation, Laguna Hills, CA, USA.10.1109/ICRA48891.2023.10160551 ↗
41. Lee, C., Sarwar, S.S., Panda, P., Srinivasan, G., and Roy, K. (2020). Enabling spike-based backpropagation for training deep neural network architectures. Front. Neurosci., 14.10.3389/fnins.2020.00119 ↗
42. Yang, Z., Wu, Y., Wang, G., Yang, Y., Li, G., Deng, L., Zhu, J., and Shi, L. (2019). DashNet: A hybrid artificial and spiking neural network for high-speed object tracking. arXiv.
43. A framework for the general design and computation of hybrid neural networks10.1038/s41467-022-30964-7 ↗
44. Aydin, A., Gehrig, M., Gehrig, D., and Scaramuzza, D. (2024, January 17–18). A hybrid ANN-SNN architecture for low-power and low-latency visual perception. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPRW63382.2024.00579 ↗
45. Advancing spiking neural networks toward deep residual learning10.1109/TNNLS.2024.3355393 ↗
46. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10–17). Swin Transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00986 ↗
47. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
48. Tran, D., Bourdev, L., Fergus, R., Torresani, L., and Paluri, M. (2015, January 7–13). Learning spatiotemporal features with 3D convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.510 ↗
49. Tran, D., Ray, J., Shou, Z., Chang, S.F., and Paluri, M. (2015). ConvNet architecture search for spatiotemporal feature learning. arXiv.
50. ReSpike: Residual frames-based hybrid spiking neural networks for efficient action recognition10.1088/2634-4386/adb070 ↗
51. Hassner, T., Itcher, Y., and Kliper-Gross, O. (2012, January 16–21). Violent flows: Real-time detection of violent crowd behavior. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA.10.1109/CVPRW.2012.6239348 ↗
52. Violence detection using oriented violent flows10.1016/j.imavis.2016.01.006 ↗