Borrow from Source Models: Efficient Infrared Object Detection with Limited Examples
Source abstract
Abstract
Recent deep models trained on large-scale RGB datasets lead to considerable achievements in visual detection tasks. However, the training examples are often limited for an infrared detection task, which may deteriorate the performance of deep detectors. In this paper, we propose a transfer approach, Source Model Guidance (SMG), where we leverage a high-capacity RGB detection model as the guidance to supervise the training process of an infrared detection network. In SMG, the foreground soft label generated from the RGB model is introduced as source knowledge to provide guidance for cross-domain transfer. Additionally, we design a Background Suppression Module in the infrared network to receive the knowledge and enhance the foreground features. SMG is easily plugged into any modern detection framework, and we show two explicit instantiations of it, SMG-C and SMG-Y, based on CenterNet and YOLOv3, respectively. Extensive experiments on different benchmarks show that both SMG-C and SMG-Y achieve remarkable performance even if the training set is scarce. Compared to advanced detectors on public FLIR, SMG-Y with 77.0% mAP outperforms others in accuracy, and SMG-C achieves real-time detection at a speed of 107 FPS. More importantly, SMG-Y trained on a quarter of the thermal dataset obtains 74.5% mAP, surpassing most state-of-the-art detectors with full FLIR as training data.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Remote-Sensing Image Classification · Infrared Target Detection Methodologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Laboratory of Infrared System Detection and Imaging Technology, Chinese Academy of Sciences, Shanghai 200083, China
Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China
University of Chinese Academy of Sciences, Beijing 100049, China
Funding
Shanghai Key Laboratory of Criminal Scene Evidence funded Foundation · Innovation Fund of Shanghai In-stitute of Technical Physics
References (40)
Show source-provided references
1. Devaguptapu, C., Akolekar, N., Sharma, M.M., and Balasubramanian, V.N. (2019, January 16–17). Borrow From Anywhere: Pseudo Multi-Modal Object Detection in Thermal Imagery. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA.10.1109/CVPRW.2019.00135 ↗
2. Zhang, L., and Peng, Z. (2019). Infrared Small Target Detection Based on Partial Sum of the Tensor Nuclear Norm. Remote Sens., 11.10.3390/rs11040382 ↗
3. Rashid, M., Khan, M.A., Alhaisoni, M., Wang, S.H., Naqvi, S.R., Rehman, A., and Saba, T. (2020). A Sustainable Deep Learning Framework for Object Recognition Using Multi-Layers Deep Features Fusion and Selection. Sustainability, 12.10.3390/su12125037 ↗
4. ImageNet Classification with Deep Convolutional Neural Networks10.1145/3065386 ↗
5. Masood, H., Zafar, A., Ali, M.U., Hussain, T., Khan, M.A., Tariq, U., and Damaševičius, R. (2022). Tracking of a Fixed-Shape Moving Object Based on the Gradient Descent Method. Sensors, 22.10.3390/s22031098 ↗
6. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
7. Intelligent Deep Learning and Improved Whale Optimization Algorithm based Framework for Object Recognition
8. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks10.1109/TPAMI.2016.2577031 ↗
9. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). SSD: Single Shot MultiBox Detector. Computer Vision—ECCV 2016, Springer International Publishing.10.1007/978-3-319-46466-4 ↗
10. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
11. Infrared aircraft few-shot classification method based on meta learning
12. The PASCAL Visual Object Classes (VOC) Challenge10.1007/s11263-009-0275-4 ↗
13. Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014). Microsoft COCO: Common Objects in Context. Computer Vision—ECCV 2014, Springer International Publishing.10.1007/978-3-319-10599-4 ↗
14. Study on an infrared multi-target detection method based on the pseudo-two-stage model10.1016/j.infrared.2021.103883 ↗
15. Miao, Z., Zhang, Y., and Li, W.H. (2021). Real-time infrared target detection based on center points. Infrared Millim. Waves, 40.
16. Konig, D., Adam, M., Jarvers, C., Layher, G., Neumann, H., and Teutsch, M. (2017, January 21–26). Fully Convolutional Region Proposal Networks for Multispectral Person Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.36 ↗
17. Liu, J., Zhang, S., Wang, S., and Metaxas, D.N. (2016). Multispectral deep neural networks for pedestrian detection. arXiv.10.5244/C.30.73 ↗
18. Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.
19. Zhou, X., Wang, D., and Krähenbühl, P. (2019). Objects as points. arXiv.
20. Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018). CornerNet: Detecting Objects as Paired Keypoints. Computer Vision—ECCV 2018, Springer International Publishing.
21. Zhou, X., Zhuo, J., and Krahenbuhl, P. (2019, January 15–20). Bottom-Up Object Detection by Grouping Extreme and Center Points. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00094 ↗
22. Tian, Z., Shen, C., Chen, H., and He, T. (November, January 27). FCOS: Fully Convolutional One-Stage Object Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.
23. Teledyne FLIR (2022, January 07). Flir Thermal Dataset for Algorithm Training [DB/OL]. FLIR. 1 September 2018., Available online: https://www.flir.com/oem/adas/adas-dataset-form/.
24. Hosang, J., Benenson, R., and Schiele, B. (2017, January 21–26). Learning Non-maximum Suppression. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.685 ↗
25. Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv.
26. Zhang, Y., Xiang, T., Hospedales, T.M., and Lu, H. (2018, January 18–23). Deep Mutual Learning. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00454 ↗
27. Huang, Z., Pan, Z., and Lei, B. (2017). Transfer Learning with Deep Convolutional Neural Network for SAR Target Classification with Limited Labeled Data. Remote Sens., 9.10.3390/rs9090907 ↗
28. Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., and Bengio, Y. (2014). Fitnets: Hints for thin deep nets. arXiv.
29. Yim, J., Joo, D., Bae, J., and Kim, J. (2017, January 21–26). A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017.10.1109/CVPR.2017.754 ↗
30. Li, Q., Jin, S., and Yan, J. (2017, January 21–26). Mimicking Very Efficient Network for Object Detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.776 ↗
31. Chen, G., Choi, W., Yu, X., Han, T., and Chandraker, M. (2017, January 4–9). Learning efficient object detection models with knowledge distillation. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
32. Chen, H., Wang, Y., Wang, G., and Qiao, Y. (2018, January 2–7). LSTD: A Low-Shot Transfer Detector for Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11716 ↗
33. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-Excitation Networks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
34. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., and Hu, Q. (2020, January 13–19). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01155 ↗
35. Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018). CBAM: Convolutional Block Attention Module. Computer Vision—ECCV 2018, Springer International Publishing.10.1007/978-3-030-01249-6 ↗
36. Zhang, Q.L., and Yang, Y.B. (2021, January 6–11). SA-Net: Shuffle Attention for Deep Convolutional Neural Networks. Proceedings of the ICASSP 2021—2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9414568 ↗
37. Wei, D., Du, Y., Du, L., and Li, L. (2021). Target Detection Network for SAR Images Based on Semi-Supervised Learning and Attention Mechanism. Remote Sens., 13.10.3390/rs13142686 ↗
38. Yu, F., Wang, D., Shelhamer, E., and Darrell, T. (2018, January 18–23). Deep Layer Aggregation. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00255 ↗
39. Zhang, S., Wen, L., Bian, X., Lei, Z., and Li, S.Z. (2018, January 18–23). Single-Shot Refinement Neural Network for Object Detection. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00442 ↗
40. Qin, P., Tang, C., Liu, Y., Zhang, J., and Xu, Z. (2021). Infrared target detection method based on improved YOLOv3. Comput. Eng., 1–12.