journal article·2025-09-30Open access

Adaptive Energy–Gradient–Contrast (EGC) Fusion with AIFI-YOLOv12 for Improving Nighttime Pedestrian Detection in Security

Lijuan Wang, Zuchao Bao, Dongming Lu

Applied Sciences· Volume 15· Issue 19

Abstract

In security applications, visible-light pedestrian detectors are highly sensitive to changes in illumination and fail under low-light or nighttime conditions, while infrared sensors, though resilient to lighting, often produce blurred object boundaries that hinder precise localization. To address these complementary limitations, we propose a practical multimodal pipeline—Adaptive Energy–Gradient–Contrast (EGC) Fusion with AIFI-YOLOv12—that first fuses infrared and low-light visible images using per-pixel weights derived from local energy, gradient magnitude and contrast measures, then detects pedestrians with an improved YOLOv12 backbone. The detector integrates an AIFI attention module at high semantic levels, replaces selected modules with A2C2f blocks to enhance cross-channel feature aggregation, and preserves P3–P5 outputs to improve small-object localization. We evaluate the complete pipeline on the LLVIP dataset and report Precision, Recall, mAP@50, mAP@50–95, GFLOPs, FPS and detection time, comparing against YOLOv8, YOLOv10–YOLOv12 baselines (n and s scales). Quantitative and qualitative results show that the proposed fusion restores complementary thermal and visible details and that the AIFI-enhanced detector yields more robust nighttime pedestrian detection while maintaining a competitive computational profile suitable for real-world security deployments.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Fire Detection and Safety Systems · Advanced Optical Sensing Technologies · Air Quality Monitoring and Forecasting

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Electronic and Optical Engineering, Nanjing University of Science and Technology Zijin College, Nanjing 210023, China

School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China

Funding

Project supported by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China

References (40)

Show source-provided references
1. Jia, X., Zhu, C., Li, M., Tang, W., Liu, S., and Zhou, W. (2021, January 11–17). LLVIP: A Visible–Infrared Paired Dataset for Low-Light Vision. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, QC, Canada.10.1109/ICCVW54120.2021.00389 ↗
2. Meng, B., Liu, H., and Ding, Z. (2025). Multi-scene Image Fusion via Memory Aware Synapses. Sci. Rep., 15.10.1038/s41598-025-88261-4 ↗
3. A review on infrared and visible image fusion algorithms based on neural networks10.1016/j.jvcir.2024.104179 ↗
4. Ma, W., Wang, K., Li, J., Yang, S.X., Li, J., Song, L., and Li, Q. (2023). Infrared and Visible Image Fusion Technology and Application: A Review. Sensors, 23.10.3390/s23020599 ↗
5. DenseFuse: A Fusion Approach to Infrared and Visible Images10.1109/TIP.2018.2887342 ↗
6. FusionGAN: A Generative Adversarial Network for Infrared and Visible Image Fusion10.1016/j.inffus.2018.09.004 ↗
7. TFDet: Target-Aware Fusion for RGB–T Pedestrian Detection10.1109/TNNLS.2024.3443455 ↗
8. Guo, H., Sun, C., Zhang, J., Zhang, W., and Zhang, N. (2024). MMYFNet: Multi-Modality YOLO Fusion Network for Object Detection in Remote Sensing Images. Remote Sens., 16.10.3390/rs16234451 ↗
9. Li, L., Shi, Y., Lv, M., Jia, Z., Liu, M., Zhao, X., Zhang, X., and Ma, H. (2024). Infrared and Visible Image Fusion via Sparse Representation and Guided Filtering in Laplacian Pyramid Domain. Remote Sens., 16.10.3390/rs16203804 ↗
10. Wang, J., Chen, Y., Sun, X., Xing, H., Zhang, F., Song, S., and Yu, S. (2025, August 23). Advancing Infrared and Visible Image Fusion with an Enhanced Multiscale Encoder and Attention-Based Networks. Information (Open Access/PMC) 2024, Available online: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11459406/.
11. GANMcC: A Generative Adversarial Network with Multi-Classification Constraints for Infrared and Visible Image Fusion
12. AttentionFGAN: Infrared and Visible Image Fusion Using Attention-Based Generative Adversarial Networks10.1109/TMM.2020.2997127 ↗
13. Ultralytics (2025). YOLOv12: Attention-Centric Real-Time Object Detectors. arXiv, Available online: https://arxiv.org/abs/2502.12524.
14. Ultralytics (2025, August 23). YOLOv12 Model Family—Documentation. Available online: https://docs.ultralytics.com/models/yolo12/.
15. Liu, R., Huang, M., Wang, L., Bi, C., and Tao, Y. (2024). PDT-YOLO: A Roadside Object-Detection Algorithm for Multiscale and Occluded Targets. Sensors, 24.10.3390/s24072302 ↗
16. Jegham, N., Koh, C.Y., Abdelatti, M., and Hendawi, A. (2024). YOLO Evolution: Comprehensive Benchmark (v3 → v12). arXiv, Available online: https://arxiv.org/abs/2411.00201.
17. Xue, Z. (2025, August 23). TFDet Code Repository. Available online: https://github.com/XueZ-phd/TFDet.
18. Qi, B., Bai, X., Wu, W., Zhang, Y., Lv, H., and Li, G. (2023). A Novel Saliency-Based Decomposition Strategy for Infrared and Visible Image Fusion. Remote Sens., 15.10.3390/rs15102624 ↗
19. He, Q., and Huang, Y. (2025). TPFusion: Texture-Preserving and Information Loss Minimization for Infrared and Visible Image Fusion. Sci. Rep., 15.10.1038/s41598-025-11482-0 ↗
20. RGB-T Image Analysis Technology and Application: A Survey10.1016/j.engappai.2023.105919 ↗
21. Zhang, H., Fromont, E., Lefèvre, S., and Avignon, B. (2021, January 3–8). Guided Attentive Feature Fusion for Multispectral Pedestrian Detection. Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACV48630.2021.00012 ↗
22. Hwang, S., Park, J., Kim, N., Choi, Y., and Kweon, I.S. (2015, January 7–12). Multispectral Pedestrian Detection: Benchmark Dataset and Baseline. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA. Available online: https://openaccess.thecvf.com/content_cvpr_2015/html/Hwang_Multispectral_Pedestrian_Detection_2015_CVPR_paper.html.10.1109/CVPR.2015.7298706 ↗
23. Zhang, L., Zhu, X., Chen, X., Yang, X., Lei, Z., and Liu, Z. (November, January 27). Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea. Available online: https://openaccess.thecvf.com/content_ICCV_2019/html/Zhang_Weakly_Aligned_Cross-Modal_Learning_for_Multispectral_Pedestrian_Detection_ICCV_2019_paper.html.10.1109/ICCV.2019.00523 ↗
24. Yang, X., Qian, Y., Zhu, H., Wang, C., and Yang, M. (2022, January 23–27). BAANet: Learning Bi-Directional Adaptive Attention Gates for Multispectral Pedestrian Detection. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.10.1109/ICRA46639.2022.9811999 ↗
25. Li, Z., Chen, Y., and Wang, L. (2024). Attention-Guided Dual-Branch Fusion Network for Infrared-Visible Pedestrian Detection in Low-Light Environments. Sensors, 24.
26. Zhang, H., Fromont, E., Lefèvre, S., and Avignon, B. (2020, January 25–28). Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blocks. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.10.1109/ICIP40778.2020.9191080 ↗
27. MS-DETR: Multispectral Pedestrian Detection Transformer with Loosely Coupled Fusion and Modality-Balanced Optimization10.1109/TITS.2024.3450584 ↗
28. Zhao, J., Wen, X., He, Y., Yang, X., and Song, K. (2024). Wavelet-Driven Multi-Band Feature Fusion for RGB-T Salient Object Detection. Sensors, 24.10.3390/s24248159 ↗
29. ICAFusion: Iterative Cross-Attention Guided Feature Fusion for Multispectral Object Detection10.1016/j.patcog.2023.109913 ↗
30. A multispectral feature fusion network for robust pedestrian detection10.1016/j.aej.2020.05.035 ↗
31. Cao, Z., Yang, H., Zhao, J., Guo, S., and Li, L. (2021). Attention Fusion for One-Stage Multispectral Pedestrian Detection. Sensors, 21.10.3390/s21124184 ↗
32. Yang, Y., Xu, K., and Wang, K. (2023). Cascaded Information Enhancement and Cross-Modal Attention Feature Fusion for Multispectral Pedestrian Detection. Front. Phys., 11.10.3389/fphy.2023.1121311 ↗
33. Fang, Q., Han, D., and Wang, Z. (2021). Cross-Modality Fusion Transformer for Multispectral Object Detection. arXiv, Available online: https://arxiv.org/abs/2111.00273.10.2139/ssrn.4227745 ↗
34. Liu, J., Zhang, H., Li, P., Wu, Y., and Wang, Y. (2022, January 10–14). Learning a Dynamic Cross-Modal Network for Multispectral Pedestrian Detection. Proceedings of the 30th ACM International Conference on Multimedia (MM ’22), Lisbon, Portugal.
35. Infrared and visible image fusion based on iterative control of anisotropic diffusion and regional gradient structure
36. Li, H., and Wu, X.-J. (2022). An improved infrared and visible image fusion using an adaptive contrast enhancement method and deep learning network with transfer learning. Remote Sens., 14.10.3390/rs14040939 ↗
37. Infrared and visible image fusion via fast approximate bilateral filter and local energy characteristics
38. Infrared and visible image fusion algorithm based on gradient attention residual dense block10.7717/peerj-cs.2569 ↗
39. Hu, K., Zhang, Q., Yuan, M., and Zhang, Y. (2024). SFDFusion: An Efficient Spatial–Frequency Domain Fusion Network for Infrared and Visible Image Fusion. arXiv, Available online: https://arxiv.org/abs/2410.22837.10.3233/FAIA240524 ↗
40. Khanam, R., and Hussain, M. (2025). A Review of YOLOv12: Attention-Based Enhancements vs. Previous Versions. arXiv, Available online: https://arxiv.org/abs/2504.11995.