Ensemble-Based Knowledge Distillation for Video Anomaly Detection
Source abstract
Abstract
Video anomaly detection has become a vital task for smart video surveillance systems because of its significant potential to minimize the video data to be analyzed by choosing unusual and critical patterns in the scenes. In this paper, we introduce three novel ensemble and knowledge distillation-based adaptive training methods to handle robust detection of different abnormal patterns in video scenes. Our approach leverages the adaptation process by providing information transfer from multiple teacher models with different network structures and further alleviates the catastrophic forgetting issue. The proposed ensemble knowledge distillation methods are implemented on two state-of-the-art anomaly detection models. We extensively evaluate our methods on two public video anomaly datasets and present a detailed analysis of our results. Finally, we show that not only does our best version model achieve comparable performance with a frame-level AUC of 75.82 to other state-of-the-art models on UCF-Crime as the target dataset, but more importantly our approaches prevent catastrophic forgetting and dramatically improve our model’s performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Human Pose and Action Recognition · Network Security and Intrusion Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Engineering, Hacettepe University, Ankara 06800, Turkey
Funding
Scientific and Technological Research Council of Turkey (TUBITAK) · Hacettepe University Scientific Research Projects Coordination Department
References (47)
Show source-provided references
1. A survey of single-scene video anomaly detection
2. Suarez, J.J.P., and Naval, P.C. (2020). A survey on deep learning techniques for video anomaly detection. arXiv.
3. Deep learning for anomaly detection: A review10.1145/3439950 ↗
4. Deep learning approaches for video-based anomalous activity detection10.1007/s11280-018-0582-1 ↗
5. Chalapathy, R., and Chawla, S. (2019). Deep learning for anomaly detection: A survey. arXiv.10.1145/3394486.3406704 ↗
6. Mohammadi, B., Fathy, M., and Sabokrou, M. (2021). Image/video deep anomaly detection: A survey. arXiv.
7. Şengönül, E., Samet, R., Abu Al-Haija, Q., Alqahtani, A., Alturki, B., and Alsulami, A.A. (2023). An Analysis of Artificial Intelligence Techniques in Surveillance Video Anomaly Detection: A Comprehensive Survey. Appl. Sci., 13.10.3390/app13084956 ↗
8. Amp-net: Appearance-motion prototype network assisted automatic video anomaly detection system10.1109/TII.2023.3298476 ↗
9. Memory-augmented appearance-motion network for video anomaly detection10.1016/j.patcog.2023.109335 ↗
10. Ren, J., Xia, F., Liu, Y., and Lee, I. (2021, January 7–10). Deep Video Anomaly Detection: Opportunities and Challenges. Proceedings of the 2021 International Conference on Data Mining Workshops (ICDMW), Auckland, New Zealand.10.1109/ICDMW53433.2021.00125 ↗
11. Analysis of anomaly detection in surveillance video: Recent trends and future vision10.1007/s11042-022-13954-1 ↗
12. Panagiotatos, G., Passalis, N., Iosifidis, A., Gabbouj, M., and Tefas, A. (2019, January 2–6). Curriculum-based teacher ensemble for robust neural network distillation. Proceedings of the 2019 27th European Signal Processing Conference (EUSIPCO), A Coruna, Spain.10.23919/EUSIPCO.2019.8903112 ↗
13. Wan, B., Fang, Y., Xia, X., and Mei, J. (2020, January 6–10). Weakly supervised video anomaly detection via center-guided discriminative learning. Proceedings of the 2020 IEEE International Conference on Multimedia and Expo (ICME), London, UK.10.1109/ICME46284.2020.9102722 ↗
14. Zhong, J.X., Li, N., Kong, W., Liu, S., Li, T.H., and Li, G. (2019, January 15–20). Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00133 ↗
15. Carreira, J., and Zisserman, A. (2017, January 21–26). Quo vadis, action recognition? A new model and the kinetics dataset. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.502 ↗
16. Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv.
17. Sultani, W., Chen, C., and Shah, M. (2018, January 18–23). Real-world anomaly detection in surveillance videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00678 ↗
18. Cheng, M., Cai, K., and Li, M. (2021, January 10–15). RWF-2000: An open large scale video database for violence detection. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.10.1109/ICPR48806.2021.9412502 ↗
19. Zhu, S., Chen, C., and Sultani, W. (2020). Computer Vision: A Reference Guide, Springer.
20. Liu, W., Luo, W., Li, Z., Zhao, P., and Gao, S. (2019, January 10–16). Margin Learning Embedded Prediction for Video Anomaly Detection with A Few Anomalies. Proceedings of the IJCAI, Macao, China.10.24963/ijcai.2019/419 ↗
21. Zhang, J., Qing, L., and Miao, J. (2019, January 22–25). Temporal convolutional network with complementary inner bag loss for weakly supervised anomaly detection. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8803657 ↗
22. Tran, D., Bourdev, L., Fergus, R., Torresani, L., and Paluri, M. (2015, January 7–13). Learning spatiotemporal features with 3d convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.510 ↗
23. Zhu, Y., and Newsam, S. (2019). Motion-aware feature for improved video anomaly detection. arXiv.
24. Ramachandra, B., Jones, M., and Vatsavai, R. (2020, January 1–5). Learning a distance function with a Siamese network to localize anomalies in videos. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass, CO, USA.10.1109/WACV45572.2020.9093417 ↗
25. Wu, P., Liu, J., Shi, Y., Sun, Y., Shao, F., Wu, Z., and Yang, Z. (2020, January 23–28). Not only look, but also listen: Learning multimodal violence detection under weak supervision. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58577-8_20 ↗
26. Zaheer, M.Z., Mahmood, A., Astrid, M., and Lee, S.I. (2020, January 23–28). Claws: Clustering assisted weakly supervised learning with normalcy suppression for anomalous event detection. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58542-6_22 ↗
27. Feng, J.C., Hong, F.T., and Zheng, W.S. (2021, January 19–25). Mist: Multiple instance self-training framework for video anomaly detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01379 ↗
28. Tian, Y., Pang, G., Chen, Y., Singh, R., Verjans, J.W., and Carneiro, G. (2021, January 11–17). Weakly-supervised video anomaly detection with contrastive learning of long and short-range temporal features. Proceedings of the 2021 18th IEEE/CVF International Conference on Computer Vision, Virtual.10.1109/ICCV48922.2021.00493 ↗
29. Wu, J., Zhang, W., Li, G., Wu, W., Tan, X., Li, Y., Ding, E., and Lin, L. (2021). Weakly-supervised spatio-temporal anomaly detection in surveillance video. arXiv.10.24963/ijcai.2021/162 ↗
30. Localizing anomalies from weakly-labeled videos10.1109/TIP.2021.3072863 ↗
31. Learning causal temporal relation and feature discrimination for anomaly detection10.1109/TIP.2021.3062192 ↗
32. Li, S., Liu, F., and Jiao, L. (March, January 22). Self-training multi-sequence learning with Transformer for weakly supervised video anomaly detection. Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22), Virtual.
33. Chen, Y., Liu, Z., Zhang, B., Fok, W., Qi, X., and Wu, Y.C. (2023, January 7–14). Mgfn: Magnitude-contrastive glance-and-focus network for weakly-supervised video anomaly detection. Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i1.25112 ↗
34. Incremental on-line learning: A review and comparison of state of the art algorithms10.1016/j.neucom.2017.06.084 ↗
35. Käding, C., Rodner, E., Freytag, A., and Denzler, J. (2016, January 20–24). Fine-tuning deep neural networks in continuous learning scenarios. Proceedings of the Asian Conference on Computer Vision, Taipei, Taiwan.10.1007/978-3-319-54526-4_43 ↗
36. Knowledge distillation: A survey10.1007/s11263-021-01453-z ↗
37. Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks10.1109/TPAMI.2021.3055564 ↗
38. Ruffy, F., and Chahal, K. (2019). The state of knowledge distillation for classification. arXiv.
39. You, S., Xu, C., Xu, C., and Tao, D. (2017, January 13–17). Learning from multiple teacher networks. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.10.1145/3097983.3098135 ↗
40. Furlanello, T., Lipton, Z., Tschannen, M., Itti, L., and Anandkumar, A. (2018, January 10–15). Born again neural networks. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
41. Park, S., and Kwak, N. (2019). Feed: Feature-level ensemble for knowledge distillation. arXiv.
42. Adaptive multi-teacher multi-level knowledge distillation10.1016/j.neucom.2020.07.048 ↗
43. Agree to disagree: Adaptive ensemble knowledge distillation in gradient space
44. Yuan, F., Shou, L., Pei, J., Lin, W., Gong, M., Fu, Y., and Jiang, D. (2021, January 2–9). Reinforced multi-teacher selection for knowledge distillation. Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, Virtual.10.1609/aaai.v35i16.17680 ↗
45. Doshi, K., and Yilmaz, Y. (2023, January 2–7). Towards interpretable video anomaly detection. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV56688.2023.00268 ↗
46. Socratic Video Understanding on Unmanned Aerial Vehicles10.1016/j.procs.2023.09.101 ↗
47. De Curtò, J., de Zarzà, I., Roig, G., and Calafate, C.T. (2023). Summarization of Videos with the Signature Transform. Electronics, 12.10.3390/electronics12071735 ↗