Adaptive Frame Sampling and Feature Alignment for Multi-Frame Infrared Small Target Detection
Source abstract
Abstract
In recent years, infrared images have attracted widespread attention, due to their extensive application in low-visibility search and rescue, forest fire monitoring, ground target monitoring, and other fields. Infrared small target detection technology plays a vital role in these applications. Although there has been significant research over the years, accurately detecting infrared small targets in complex backgrounds remains a significant challenge. Multi-frame detection methods can significantly improve detection performance in these cases. However, current multi-frame methods face difficulties in balancing the number of input frames and detection speed, and cannot effectively handle the background motion caused by movement of the infrared camera. To address these issues, we propose an adaptive frame sampling method and a detection network aligned at the feature level. Our adaptive frame sampling method uses mutual information to measure motion changes between adjacent frames, construct a motion distribution, and sample frames with uniform motion based on the averaged motion distribution. Our detection network handles background motion by predicting a homography flow matrix that aligns features at the feature level. Extensive evaluation of all components showed that the proposed method can more effectively perform multi-frame infrared small target detection.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Infrared Target Detection Methodologies · Advanced Measurement and Detection Methods · Thermography and Photoacoustic Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Automation Department, School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, China
Funding
National Natural Science Foundation of China (NSFC)
References (48)
Show source-provided references
1. A kernel-based nonparametric regression method for clutter removal in infrared small-target detection applications10.1109/LGRS.2009.2039192 ↗
2. Infrared dim and small target detection based on stable multisubspace learning in heterogeneous scene10.1109/TGRS.2017.2709250 ↗
3. Real-time visual enhancement for infrared small dim targets in video10.1016/j.infrared.2017.05.002 ↗
4. Infrared small target detection via self-regularized weighted sparse model10.1016/j.neucom.2020.08.065 ↗
5. Single frame infrared image small target detection via patch similarity propagation based background estimation10.1016/j.infrared.2020.103197 ↗
6. Low-rank approximation and multiple sparse constraint modeling for infrared low-flying fixed-wing UAV detection10.1109/JSTARS.2021.3069032 ↗
7. The design of top-hat morphological filter and application to infrared target detection10.1016/j.infrared.2005.04.006 ↗
8. Regularized background adaptation: A novel learning rate control scheme for Gaussian mixture modeling
9. Analysis of selection of structural element in mathematical morphology with application to infrared point target detection
10. Enhanced video-based target detection using multi-frame correlation filtering10.1109/TAES.2009.4805280 ↗
11. Space moving target detection and tracking method in complex background10.1016/j.infrared.2018.03.007 ↗
12. Du, J., Li, D., Deng, Y., Zhang, L., Lu, H., Hu, M., Shen, X., Liu, Z., and Ji, X. (2021, January 22–24). Multiple frames based infrared small target detection method using CNN. Proceedings of the 2021 4th International Conference on Algorithms, Computing and Artificial Intelligence, Sanya, China.10.1145/3508546.3508607 ↗
13. A spatial-temporal feature-based detection framework for infrared dim small target
14. Boost infrared moving aircraft detection performance by using fast homography estimation and dual input object detection network10.1016/j.infrared.2022.104182 ↗
15. Infrared patch-image model for small target detection in a single image10.1109/TIP.2013.2281420 ↗
16. A robust infrared small target detection algorithm based on human visual system10.1109/LGRS.2014.2323236 ↗
17. Multiscale patch-based contrast measure for small infrared target detection10.1016/j.patcog.2016.04.002 ↗
18. Infrared small-target detection using multiscale gray difference weighted image entropy10.1109/TAES.2015.140878 ↗
19. Infrared small target and background separation via column-wise weighted robust principal component analysis10.1016/j.infrared.2016.06.021 ↗
20. Target detection algorithm based on two layers human visual system10.3390/a8030541 ↗
21. Image small target detection based on deep learning with SNR controlled sample generation
22. Infrared dim and small target detection based on two-stage U-skip context aggregation network with a missed-detection-and-false-alarm combination loss10.1007/s11042-019-7643-z ↗
23. Interior attention-aware network for infrared small target detection
24. Dense nested attention network for infrared small target detection10.1109/TIP.2022.3199107 ↗
25. Infrared moving targets detection based on optical flow estimation
26. Sparse representation based infrared small target detection via an online-learned double sparse background dictionary10.1016/j.infrared.2019.04.001 ↗
27. Infrared moving small-target detection via spatiotemporal consistency of trajectory points10.1109/LGRS.2019.2916642 ↗
28. Infrared small target detection using nonoverlapping patch spatial–temporal tensor factorization with capped nuclear norm regularization
29. Practical approaches to target detection in long range and low quality infrared videos10.5121/sipij.2021.12301 ↗
30. Target Detection and Classification Performance Enhancement Using Superresolution Infrared Videos10.5121/sipij.2021.12203 ↗
31. Local motion and contrast priors driven deep network for infrared small target superresolution10.1109/JSTARS.2022.3183230 ↗
32. Yao, S., Zhu, Q., Zhang, T., Cui, W., and Yan, P. (2022). Infrared image small-target detection based on improved FCOS and spatio-temporal features. Electronics, 11.10.3390/electronics11060933 ↗
33. Sun, J., Wei, M., Wang, J., Zhu, M., Lin, H., Nie, H., and Deng, X. (2024). CenterADNet: Infrared Video Target Detection Based on Central Point Regression. Sensors, 24.10.3390/s24061778 ↗
34. Multimodality image registration by maximization of mutual information10.1109/42.563664 ↗
35. Zhi, Y., Tong, Z., Wang, L., and Wu, G. (2021, January 11–17). Mgsampler: An explainable sampling strategy for video action recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00154 ↗
36. Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (November, January 27). Searching for mobilenetv3. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
37. Andrew, G., and Menglong, Z. (2017). Efficient convolutional neural networks for mobile vision applications, mobilenets. arXiv.
38. Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., and Liu, W. (November, January 27). Ccnet: Criss-cross attention for semantic segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
39. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
40. Hong, M., Lu, Y., Ye, N., Lin, C., Zhao, Q., and Liu, S. (2022, January 21–24). Unsupervised homography estimation with coplanarity-aware gan. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01714 ↗
41. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
42. Hara, K., Kataoka, H., and Satoh, Y. (2017, January 22–29). Learning spatio-temporal features with 3d residual networks for action recognition. Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, Italy.10.1109/ICCVW.2017.373 ↗
43. Liu, Z., Wang, L., Wu, W., Qian, C., and Lu, T. (2021, January 11–17). Tam: Temporal adaptive module for video recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.01345 ↗
44. Zhang, J., Wang, C., Liu, S., Jia, L., Ye, N., Wang, J., Zhou, J., and Sun, J. (2020, January 23–28). Content-aware unsupervised deep homography estimation. Proceedings of the Computer Vision—ECCV 2020: 16th European Conference, Glasgow, UK. Part I 16.10.1007/978-3-030-58452-8_38 ↗
45. A dataset for infrared detection and tracking of dim-small aircraft targets under ground/air background
46. Infrared small target detection utilizing the multiscale relative local contrast measure10.1109/LGRS.2018.2790909 ↗
47. Detecting slowly moving infrared targets using temporal filtering and association strategy10.1631/FITEE.1601203 ↗
48. Real-time detection and tracking of infrared small targets based on grid fast density peaks searching and improved KCF10.1016/j.infrared.2022.104181 ↗