Automatic Handgun Detection with Deep Learning in Video Surveillance Images
Source abstract
Abstract
There is a great need to implement preventive mechanisms against shootings and terrorist acts in public spaces with a large influx of people. While surveillance cameras have become common, the need for monitoring 24/7 and real-time response requires automatic detection methods. This paper presents a study based on three convolutional neural network (CNN) models applied to the automatic detection of handguns in video surveillance images. It aims to investigate the reduction of false positives by including pose information associated with the way the handguns are held in the images belonging to the training dataset. The results highlighted the best average precision (96.36%) and recall (97.23%) obtained by RetinaNet fine-tuned with the unfrozen ResNet-50 backbone and the best precision (96.23%) and F1 score values (93.36%) obtained by YOLOv3 when it was trained on the dataset including pose information. This last architecture was the only one that showed a consistent improvement—around 2%—when pose information was expressly considered during training.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Advanced Neural Network Applications · Advanced Optical Sensing Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical, Electronic, Automatic and Communications Engineering—IEEAC, School of Computer Science, University of Castilla-La Mancha, Paseo de la Universidad 4, 13071 Ciudad Real, Spain
Department of Electrical, Electronic, Automatic and Communications Engineering—IEEAC, Higher Technical School of Industrial Engineering, University of Castilla-La Mancha, Avenida de Camilo José Cela s/n, 13071 Ciudad Real, Spain
Funding
Ministerio de Economía, Industria y Competitividad, Gobierno de España
References (50)
Show source-provided references
1. Karp, A. (2021, March 04). Estimating Global Civilian-Held Firearms Numbers. Briefing Paper in Small Arms Survey. Available online: http://www.smallarmssurvey.org/.
2. Spagnolo, P., Mazzeo, P.L., and Distante, C. (2014). Human Behavior Understanding in Networked Sensing, Springer International Publishing.10.1007/978-3-319-10807-0 ↗
3. Real-time smart surveillance using motion analysis10.1111/j.1468-0394.2010.00545.x ↗
4. Deep learning assisted portable IR active imaging sensor spots and identifies live humans through fire10.1016/j.optlaseng.2019.105818 ↗
5. Xiao, Z., Lu, X., Yan, J., Wu, L., and Ren, L. (2015, January 16–18). Automatic detection of concealed pistols using passive millimeter wave imaging. Proceedings of the 2015 IEEE International Conference on Imaging Systems and Techniques (IST), Macau, China.10.1109/IST.2015.7294538 ↗
6. A Computer Vision based Framework for Visual Gun Detection Using Harris Interest Point Detector10.1016/j.procs.2015.06.083 ↗
7. Grega, M., Matiolański, A., Guzik, P., and Leszczuk, M. (2016). Automated Detection of Firearms and Knives in a CCTV Image. Sensors, 16.10.3390/s16010047 ↗
8. Sultana, F., Sufian, A., and Dutta, P. (2020). A Review of Object Detection Models Based on Convolutional Neural Network. Advances in Intelligent Systems and Computing, Springer.10.1007/978-981-15-4288-6_1 ↗
9. A survey of the recent architectures of deep convolutional neural networks10.1007/s10462-020-09825-6 ↗
10. Object Detection With Deep Learning: A Review10.1109/TNNLS.2018.2876865 ↗
11. Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review10.1162/neco_a_00990 ↗
12. Goodfellow, I., Bengio, J., Courville, A., and Bach, F. (2016). Deep Learning, MIT Press Ltd.
13. Khan, M.A., and Kim, J. (2020). Toward Developing Efficient Conv-AE-Based Intrusion Detection System Using Heterogeneous Dataset. Electronics, 9.10.3390/electronics9111771 ↗
14. The Pascal Visual Object Classes (VOC) Challenge10.1007/s11263-009-0275-4 ↗
15. Microsoft COCO: Common Objects in Context10.1007/978-3-319-10602-1_48 ↗
16. Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., and Kolesnikov, A. (2018). The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale. arXiv.
17. Padilla, R., Passos, W.L., Dias, T.L.B., Netto, S.L., and da Silva, E.A.B. (2021). A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit. Electronics, 10.10.3390/electronics10030279 ↗
18. Padilla, R., Netto, S.L., and da Silva, E.A.B. (2020, January 1–3). A Survey on Performance Metrics for Object-Detection Algorithms. Proceedings of the 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), Niteroi, Brazil.10.1109/IWSSIP48289.2020.9145130 ↗
19. Gelana, F., and Yadav, A. (2018). Firearm Detection from Surveillance Cameras Using Image Processing and Machine Learning Techniques. Smart Innovations in Communication and Computational Sciences, Springer.10.1007/978-981-13-2414-7_3 ↗
20. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
21. Selective Search for Object Recognition10.1007/s11263-013-0620-5 ↗
22. Elsner, J., Fritz, T., Henke, L., Jarrousse, O., Taing, S., and Uhlenbrock, M. (2018). Automatic Weapon Detection in Social Media Image Data Using a Two-Pass Convolutional Neural Network, European Law Enforcement Research Bulletin. (4 SCE).
23. Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.169 ↗
24. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks10.1109/TPAMI.2016.2577031 ↗
25. Verma, G.K., and Dhillon, A. (2017, January 24–26). A Handheld Gun Detection using Faster R-CNN Deep Learning. Proceedings of the 7th International Conference on Computer and Communication Technology-ICCCT-2017, Allahabad, India.10.1145/3154979.3154988 ↗
26. Automatic handgun detection alarm in videos using deep learning10.1016/j.neucom.2017.05.012 ↗
27. Real-time gun detection in CCTV: An open problem10.1016/j.neunet.2020.09.013 ↗
28. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
29. Redmon, J., and Farhadi, A. (2018). YOLOv3: An incremental improvement. arXiv.
30. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). SSD: Single Shot MultiBox Detector. Computer Vision–ECCV 2016, Springer.10.1007/978-3-319-46448-0_2 ↗
31. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollar, P. (2017, January 22–29). Focal Loss for Dense Object Detection. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.324 ↗
32. Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv.
33. Romero, D., and Salamea, C. (2019). Convolutional Models for the Detection of Firearms in Surveillance Videos. Appl. Sci., 9.10.3390/app9152965 ↗
34. Kanehisa, R., and Neto, A. (2019, January 19–21). Firearm Detection using Convolutional Neural Networks. Proceedings of the 11th International Conference on Agents and Artificial Intelligence-Volume 2: ICAART, Prague, Czech Republic. INSTICC.10.5220/0007397707070714 ↗
35. Warsi, A., Abdullah, M., Husen, M.N., Yahya, M., Khan, S., and Jawaid, N. (2019, January 27–29). Gun Detection System Using YOLOv3. Proceedings of the 2019 IEEE International Conference on Smart Instrumentation, Measurement and Application (ICSIMA), Kuala Lumpur, Malaysia.10.1109/ICSIMA47653.2019.9057329 ↗
36. In object detection deep learning methods, YOLO shows supremum to Mask R-CNN10.1088/1742-6596/1529/4/042086 ↗
37. Warsi, A., Abdullah, M., Husen, M.N., and Yahya, M. (2020, January 3–5). Automatic Handgun and Knife Detection Algorithms: A Review. Proceedings of the 2020 14th International Conference on Ubiquitous Information Management and Communication (IMCOM), Taichung, Taiwan.10.1109/IMCOM48794.2020.9001725 ↗
38. Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
39. He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.10.1109/CVPR.2016.90 ↗
40. Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature Pyramid Networks for Object Detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
41. M2Det: A Single-Shot Object Detector Based on Multi-Level Feature Pyramid Network
42. Elmir, Y., Laouar, S.A., and Hamdaoui, L. (2019, January 27). Deep Learning for Automatic Detection of Handguns in Video Sequences. Proceedings of the 3rd edition of the National Study Day on Research on Computer Sciences (JERI 2019), Saida, Algeria.
43. Lim, J., Jobayer, M.I.A., Baskaran, V.M., Lim, J.M., Wong, K., and See, J. (2019, January 18–21). Gun Detection in Surveillance Videos using Deep Neural Networks. Proceedings of the 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Lanzhou, China.10.1109/APSIPAASC47483.2019.9023182 ↗
44. Velasco Mata, A. (2020). Human Pose Information as an Improvement Factor for Handgun Detection. [Master’s Thesis, Escuela Superior de Informática, Univ. de Castilla-La Mancha].
45. OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields10.1109/TPAMI.2019.2929257 ↗
46. Xu, Y. (2021, June 29). Faster R-CNN (Object Detection) Implemented by Keras for Custom Data from Google’s Open Images . . . . Towards Data Science. Available online: https://towardsdatascience.com/faster-r-cnn-object-detection-implemented-by-keras-for-custom-data-from-googles-open-images-125f62b9141a.
47. Gaiser, H., Vries, M.D., Lacatusu, V., Williamson, A., Liscio, E., Henon, Y., and Gratie, C. (2021, March 04). fizyr/kerasretinanet 0.5.1. 2019. GitHub. Available online: https://github.com/fizyr/keras-retinanet.
48. Balsys, R. (2021, March 18). pythonlessons/YOLOv3-Object-Detection-Tutorial. Available online: https://pylessons.com/YOLOv3-TF2-introduction/.
49. Darrenl (2021, March 04). Tzutalin/Labelimg. 2018. GitHub. Available online: https://github.com/tzutalin/labelImg.
50. Lee, K., Lee, I., and Lee, S. (2018). Propagating LSTM: 3D Pose Estimation Based on Joint Interdependency. Computer Vision–ECCV 2018, Springer.10.1007/978-3-030-01234-2_8 ↗