Multiscale Object Detection in Infrared Streetscape Images Based on Deep Learning and Instance Level Data Augmentation
Source abstract
Abstract
The development of object detection in infrared images has attracted more attention in recent years. However, there are few studies on multi-scale object detection in infrared street scene images. Additionally, the lack of high-quality infrared datasets hinders research into such algorithms. In order to solve these issues, we firstly make a series of modifications based on Faster Region-Convolutional Neural Network (R-CNN). In this paper, a double-layer region proposal network (RPN) is proposed to predict proposals of different scales on both fine and coarse feature maps. Secondly, a multi-scale pooling module is introduced into the backbone of the network to explore the response of objects on different scales. Furthermore, the inception4 module and the position sensitive region of interest (ROI) align (PSalign) pooling layer are utilized to explore richer features of the objects. Thirdly, this paper proposes instance level data augmentation, which takes into account the imbalance between categories while enlarging dataset. In the training stage, the online hard example mining method is utilized to further improve the robustness of the algorithm in complex environments. The experimental results show that, compared with baseline, our detection method has state-of-the-art performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Video Surveillance and Tracking Methods · Remote-Sensing Image Classification
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China
Funding
National Natural Science Foundation of China
References (32)
Show source-provided references
1. Detection of pedestrians in far-infrared automotive night vision using region-growing and clothing distortion compensation10.1016/j.infrared.2010.09.006 ↗
2. Qi, B., John, V., Liu, Z., and Mita, S. (2014, January 23–28). Use of Sparse Representation for Pedestrian Detection in Thermal Images. Proceedings of the Workshop on Perception Beyond the Visible Spectrum, Columbus, OH, USA.10.1109/CVPRW.2014.49 ↗
3. Bertozzi, M., Broggi, A., Carletti, M., Fascioli, A., Graf, T., Grisleri, P., and Meinecke, M. (2003, January 10–12). IR Pedestrian Detection for Advanced Driver Assistance Systems. Proceedings of the Pattern Recognition, Dagm Symposium, Magdeburg, Germany.10.1007/978-3-540-45243-0_74 ↗
4. Pedestrian detection and tracking in infrared imagery using shape and appearance10.1016/j.cviu.2006.08.009 ↗
5. Fast Feature Pyramids for Object Detection10.1109/TPAMI.2014.2300479 ↗
6. Biswas, S.K., and Milanfar, P. (2016). Linear Support Tensor Machine: Pedestrian Detection in Thermal Infrared Images. IEEE Trans. Image Process.10.1109/TIP.2017.2705426 ↗
7. Algorithm for Infrared Pedestrian Detection Based on Aggregated Channel Features
8. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
9. Girshick, R. (arXiv, 2015). Fast R-CNN, arXiv.10.1109/ICCV.2015.169 ↗
10. Hou, Y.L., Song, Y., Hao, X., Shen, Y., and Qian, M. (2017, January 22–25). Multispectral pedestrian detection based on deep convolutional neural networks. Proceedings of the IEEE International Conference on Signal Processing, Communications and Computing, Xiamen, China.10.1109/ICSPCC.2017.8242507 ↗
11. Liu, J., Zhang, S., Wang, S., and Metaxas, D.N. (arXiv, 2016). Multispectral deep neural networks for pedestrian detection, arXiv.10.5244/C.30.73 ↗
12. Galarza-Bravo, M.A., and Flores-Calero, M.J. (2018, January 9–11). Pedestrian Detection at Night Based on Faster R-CNN and Far Infrared Images. Proceedings of the International Conference on Intelligent Robotics and Applications, Newcastle, NSW, Australia.10.1007/978-3-319-97589-4_28 ↗
13. Kong, T., Yao, A., Chen, Y., and Sun, F. (2016, January 27–30). HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.98 ↗
14. Ren, Y., Zhu, C., and Xiao, S. (2018). Small Object Detection in Optical Remote Sensing Images via Modified Faster R-CNN. Appl. Sci., 8.10.3390/app8050813 ↗
15. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (arXiv, 2015). SSD: Single Shot MultiBox Detector, arXiv.10.1007/978-3-319-46448-0_2 ↗
16. Jeong, J., Park, H., and Kwak, N. (arXiv, 2017). Enhancement of SSD by concatenating feature maps for object detection, arXiv.10.5244/C.31.76 ↗
17. Fu, C.Y., Liu, W., Ranga, A., Tyagi, A., and Berg, A.C. (arXiv, 2017). DSSD: Deconvolutional Single Shot Detector, arXiv.
18. Lin, T.-Y., Dollár, P., Girshick, R.B., He, K., Hariharan, B., and Belongie, S.J. (2017, January 21–26). Feature Pyramid Networks for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
19. Kong, T., Sun, F., Yao, A., Liu, H., Lu, M., and Chen, Y. (2017, January 21–26). Ron: Reverse connection with objectness prior networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.557 ↗
20. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask r-cnn. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.322 ↗
21. Zhong, Z., Jin, L., Zhang, S., and Feng, Z. (arXiv, 2016). Deeptext: A unified framework for text proposal generation and text detection in natural images, arXiv.10.1109/ICASSP.2017.7952348 ↗
22. Xu, D., Ouyang, W., Ricci, E., Wang, X., and Sebe, N. (arXiv, 2017). Learning Cross-Modal Deep Representations for Robust Pedestrian Detection, arXiv.10.1109/CVPR.2017.451 ↗
23. Yang, X., Sun, H., Fu, K., Yang, J., Sun, X., Yan, M., and Guo, Z. (2018). Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks. Remote Sens., 10.10.3390/rs10010132 ↗
24. Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y., and Sun, J. (arXiv, 2018). DetNet: A Backbone network for Object Detection, arXiv.10.1007/978-3-030-01240-3_21 ↗
25. Liu, S., Huang, D., and Wang, Y. (arXiv, 2018). Receptive Field Block Net for Accurate and Fast Object Detection, arXiv.10.1007/978-3-030-01252-6_24 ↗
26. Zhong, Z., Zheng, L., Kang, G., Li, S., and Yang, Y. (arXiv, 2017). Random Erasing Data Augmentation, arXiv.
27. Exploratory undersampling for class-imbalance learning10.1109/TSMCB.2008.2007853 ↗
28. Szegedy, C., Ioffe, S., and Vanhoucke, V. (arXiv, 2016). Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning, arXiv.10.1609/aaai.v31i1.11231 ↗
29. Dai, J., Li, Y., He, K., and Sun, J. (arXiv, 2016). R-FCN: Object Detection via Region-based Fully Convolutional Networks, arXiv.
30. Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y., and Sun, J. (arXiv, 2017). Light-Head R-CNN: In Defense of Two-Stage Object Detector, arXiv.
31. Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks
32. Zhu, C., Zheng, Y., Luu, K., and Savvides, M. (arXiv, 2016). CMS-RCNN: Contextual Multi-Scale Region-Based CNN for Unconstrained Face Detection, arXiv.10.1007/978-3-319-61657-5_3 ↗