Multi-Target Detection Algorithm for Fusion Images Based on an Attention Mechanism
Source abstract
Abstract
Due to the inherent limitations of visible-light sensors, monitoring systems that rely solely on single-modal visible-light images exhibit reduced accuracy, posing safety concerns in applications such as autonomous driving. Infrared and visible-light image fusion technology addresses this issue by generating composite images that integrate complementary information from both modalities, thereby enhancing perception robustness. This study focuses on target detection in fused images. Given that targets in such images are often small and severely occluded, we propose an optimized detection framework to overcome these challenges. Specifically, we improve the YOLOv8 baseline model by introducing a dedicated small-object detection layer, incorporating the Global Attention Mechanism (GAM), and refining the loss function. Experimental results show that our method achieves a 5.0% improvement in mAP and a 6.5% gain in recall over the original YOLOv8. Furthermore, comparative experiments on fused and single-modal inputs demonstrate that fused images yield the highest detection accuracy. These results confirm that leveraging fused inputs significantly enhances detection accuracy and robustness in complex environments.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Fusion Techniques · Infrared Target Detection Methodologies · Medical Image Segmentation Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China
Institute of Systems Engineering, China Academy of Engineering Physics, Mianyang 621999, China
References (27)
Show source-provided references
1. Anis, M. (2025, June 14). Integrating Visible-Infrared Fusion and Deep Learning for Enhanced Object Detection in Complex Environments. Available online: https://www.researchgate.net/publication/384994474_Integrating_Visible-Infrared_Fusion_and_Deep_Learning_for_Enhanced_Object_Detection_in_Complex_Environments.
2. Ma, W., Wang, K., Li, J., Yang, S.X., Li, J., Song, L., and Li, Q. (2023). Infrared and Visible Image Fusion Technology and Application: A Review. Sensors, 23.10.3390/s23020599 ↗
3. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
4. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
5. Hwang, S., Park, J., Kim, N., Choi, Y., and Kweon, I.S. (2015, January 7–12). Multispectral Pedestrian Detection: Benchmark Dataset and Baseline. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298706 ↗
6. M2FNet: Multi-Modal Fusion Network for Object Detection from Visible and Thermal Infrared Images
7. Infrared and Visible Image Fusion Methods and Applications: A Survey10.1016/j.inffus.2018.02.004 ↗
8. Liu, Y., Dong, L., Ji, Y., and Xu, W. (2019). Infrared and Visible Image Fusion through Details Preservation. Sensors, 19.10.3390/s19204556 ↗
9. Multi-Scale Convolutional Neural Networks and Saliency Weight Maps for Infrared and Visible Image Fusion10.1016/j.jvcir.2023.104015 ↗
10. MrFDDGAN: Multi-Receptive Field Feature Transfer and Dual Discriminator-Driven Generative Adversarial Network for Infrared and Color Visible Image Fusion
11. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
12. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C., and Berg, A.C. (2016). SSD: Single Shot MultiBox Detector. European Conference on Computer Vision (ECCV), Springer.10.1007/978-3-319-46448-0_2 ↗
13. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal Loss for Dense Object Detection. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.324 ↗
14. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-Excitation Networks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
15. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018). CBAM: Convolutional Block Attention Module. European Conference on Computer Vision, Springer.10.1007/978-3-030-01234-2_1 ↗
16. Park, J., Woo, S., Lee, J.Y., and Kweon, I.S. (2018). BAM: Bottleneck Attention Module. arXiv.
17. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020). End-to-End Object Detection with Transformers. Proceedings Europe Conference Computer Vision (ECCV), Springer.10.1007/978-3-030-58452-8_13 ↗
18. Liu, Y., Shao, Z., and Hoffmann, N. (2021). Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions. arXiv.
19. The TNO Multiband Image Data Collection10.1016/j.dib.2017.09.038 ↗
20. U2Fusion: A Unified Unsupervised Image Fusion Network10.1109/TPAMI.2020.3012548 ↗
21. Takumi, K., Watanabe, K., Ha, Q., Tejero-De-Pablos, A., Ushiku, Y., and Harada, T. (2017, January 23–27). Multispectral Object Detection for Autonomous Vehicles. Proceedings of the Thematic Workshops of ACM Multimedia, Mountain View, CA, USA.10.1145/3126686.3126727 ↗
22. Liu, J., Fan, X., Huang, Z., Wu, G., Liu, R., Zhong, W., and Luo, Z. (, January 18–24). Target-Aware Dual Adversarial Learning and a Multi-Scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00571 ↗
23. Jia, X., Zhu, C., Li, M., Tang, W., and Zhou, W. (2021, January 11–17). LLVIP: A Visible-Infrared Paired Dataset for Low-Light Vision. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCV), Montreal, BC, Canada.10.1109/ICCVW54120.2021.00389 ↗
24. Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21–26). Image-to-Image Translation with Conditional Adversarial Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.632 ↗
25. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
26. Zhao, Z., Xu, S., Zhang, C., Liu, J., Li, P., and Zhang, J. (2020). DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion. arXiv.10.24963/ijcai.2020/135 ↗
27. Tong, Z., Chen, Y., Xu, Z., and Yu, R. (2023). Wise-IoU: Bounding Box Regression Loss with Dynamic Focusing Mechanism. arXiv.