Deep Feature Fusion Based Dual Branch Network for X-ray Security Inspection Image Classification
Source abstract
Abstract
Automatic computer security inspection of X-ray scanned images has an irresistible trend in modern life. Aiming to address the inconvenience of recognizing small-sized prohibited item objects, and the potential class imbalance within multi-label object classification of X-ray scanned images, this paper proposes a deep feature fusion model-based dual branch network architecture. Firstly, deep feature fusion is a method to fuse features extracted from several model layers. Specifically, it operates these features by upsampling and dimension reduction to match identical sizes, then fuses them by element-wise sum. In addition, this paper introduces focal loss to handle class imbalance. For balancing importance on samples of minority and majority class, it assigns weights to class predictions. Additionally, for distinguishing difficult samples from easy samples, it introduces modulating factor. Dual branch network adopts the two components above and integrates them in final loss calculation through the weighted sum. Experimental results illustrate that the proposed method outperforms baseline and state-of-art by a large margin on various positive/negative ratios of datasets. These demonstrate the competitivity of the proposed method in classification performance and its potential application under actual circumstances.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Industrial Vision Systems and Defect Detection · Image and Object Detection Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, China
University of Chinese Academy of Sciences, Beijing 100049, China
Funding
Special Fund for Basic Research on Scientific Instruments of the National Natural Science Foundation of China · Science and Technology Innovation Plan Of Shanghai Science and Technology Commission
References (39)
Show source-provided references
1. Recent progress on X-ray security inspection technologies10.1360/N972016-00698 ↗
2. Mery, D. (2013, January 23–28). X-ray testing by computer vision. Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA.10.1109/CVPRW.2013.61 ↗
3. Deep learning10.1038/nature14539 ↗
4. Hyperspectral Image Classification with Deep Feature Fusion Network10.1109/TGRS.2018.2794326 ↗
5. Deep learning for pixel-level image fusion: Recent advances and future prospects10.1016/j.inffus.2017.10.007 ↗
6. Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the 2017 IEEE International Conference on Computer Vision, Honolulu, HI, USA.10.1109/ICCV.2017.324 ↗
7. Berciano, A., Díaz-Pernil, D., Kropatsch, W., Molina-Abril, H., and Real, P. (2011). Visual words on baggage X-ray images. Computer Analysis of Images and Patterns, Proceedings of the 2011 International Conference on Computer Analysis of Images and Patterns, Seville, Spain, 29–31 August 2011, Springer.
8. Turcsany, D., Mouton, A., and Breckon, T.P. (2013, January 25–28). Improving feature-based object recognition for X-ray baggage security screening using primed visual words. Proceedings of the 2013 IEEE International Conference on Industrial Technology, Portland, OR, USA.10.1109/ICIT.2013.6505833 ↗
9. Kundegorski, M.E., Akçay, S., Devereux, M., Mouton, A., and Breckon, T.P. (2016, January 23–25). On using feature descriptors as visual words for object detection within X-ray baggage security screening. Proceedings of the 7th International Conference on Imaging for Crime Detection and Prevention, London, UK.10.1049/ic.2016.0080 ↗
10. Support vector machines10.1109/5254.708428 ↗
11. Akcay, S., and Breckon, T. (2020). Towards automatic threat detection: A survey of advances of deep learning within X-ray security imaging. arXiv.
12. Bräunl, T., McCane, B., Rivera, M., and Yu, X. (2015). Object recognition in baggage inspection using adaptive sparse representations of X-ray images. Image and Video Technology, Proceedings of the 2015 Pacific-Rim Symposium on Image and Video Technology, Auckland, New Zealand, 25–27 November 2015, Springer.10.1007/978-3-319-29451-3 ↗
13. Multi-targeted backdoor: Indentifying backdoor attack for multiple deep neural networks10.1587/transinf.2019EDL8170 ↗
14. GDXray: The database of X-ray images for nondestructive testing10.1007/s10921-015-0315-7 ↗
15. Using Deep Convolutional Neural Network Architectures for Object Classification and Detection Within X-Ray Baggage Security Imagery10.1109/TIFS.2018.2812196 ↗
16. An evaluation of deep learning-based object detection strategies for threat object detection in baggage security imagery10.1016/j.patrec.2019.01.014 ↗
17. X-ray security inspection method using active vision based on Q-learning algorithm
18. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Li, F.-F. (2009, January 20–25). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
19. Region-Based Convolutional Networks for Accurate Object Detection and Segmentation10.1109/TPAMI.2015.2437384 ↗
20. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
21. Miao, C., Xie, L., Wan, F., Su, C., Liu, H., Jiao, J., and Ye, Q. (2019, January 15–20). SIXray: A large-scale security inspection X-ray benchmark for prohibited item discovery in overlapping images. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00222 ↗
22. Rong, T., Cai, H., and Xiong, Y. (2021). An Enhanced Prohibited Items Recognition Model. arXiv.
23. Hu, B., Zhang, C., Wang, L., Zhang, Q., and Liu, Y. (December, January 30). Multi-label X-ray Imagery Classification via Bottom-up Attention and Meta Fusion. Proceedings of the Asian Conference on Computer Vision, Tokyo, Japan.
24. Hassan, T., Shafay, M., Akçay, S., Khan, S., Bennamoun, M., Damiani, E., and Werghi, N. (2020). Meta-Transfer Learning Driven Tensor-Shot Detector for the Autonomous Localization and Recognition of Concealed Baggage Threats. Sensors, 20.10.3390/s20226450 ↗
25. Dumagpi, J.K., and Jeong, Y.J. (2021). Evaluating GAN-Based Image Augmentation for Threat Detection in Large-Scale Xray Security Images. Appl. Sci., 11.10.3390/app11010036 ↗
26. Wei, Y., Tao, R., Wu, Z., Ma, Y., Zhang, L., and Liu, X. (2020, January 12–16). Occluded prohibited items detection: An x-ray security inspection benchmark and de-occlusion attention module. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.10.1145/3394171.3413828 ↗
27. Deep Feature Fusion for VHR Remote Sensing Scene Classification10.1109/TGRS.2017.2700322 ↗
28. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016, January 8–16). SSD: Single shot multibox detector. Proceedings of the 2016 European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
29. Eigen, D., and Fergus, R. (2015, January 7–13). Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. Proceedings of the 2015 IEEE International Conference on Computer Vision, Boston, MA, USA.10.1109/ICCV.2015.304 ↗
30. Li, Z., and Zhou, F. (2017). FSSD: Feature fusion single shot multibox detector. arXiv.
31. Zhang, Z., Zhang, X., Peng, C., Xue, X., and Sun, J. (2018, January 8–14). ExFuse: Enhancing feature fusion for semantic segmentation. Proceedings of the 2018 European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01249-6_17 ↗
32. Methods of training data augmentation for medical image artificial intelligence aided diagnosis
33. Shrivastava, A., Gupta, A., and Girshick, R. (2016, January 27–30). Training region-based object detectors with online hard example mining. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.89 ↗
34. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
35. Zeiler, M.D., Krishnan, D., Taylor, G.W., and Fergus, R. (2010, January 13–18). Deconvolutional networks. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5539957 ↗
36. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., and Wang, Z. (2016, January 27–30). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.207 ↗
37. Bottou, L. (2010). Large-scale machine learning with stochastic gradient descent. COMPSTAT’2010, Proceedings of the 19th International Conference on Computational Statistics, Paris, France, 22–27 August 2010, Physica-Verlag HD.10.1007/978-3-7908-2604-3_16 ↗
38. The pascal Visual Object Classes (VOC) challenge10.1007/s11263-009-0275-4 ↗
39. Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv.