Thermal Depth Estimation Using Unified Multi-Scale Features and Propagation-Based Refinement
Source abstract
Abstract
Thermal monocular depth estimation can provide more robust depth predictions than RGB-based methods under nighttime and adverse weather conditions. However, when trained with projected LiDAR supervision, depth models often retain structural errors in sky regions, long-range areas, and object boundaries because LiDAR measurements are sparse or missing in such regions. To address this limitation, we propose a thermal monocular depth estimation framework that incorporates propagation-based refinement. To make this refinement applicable across different base models, we further design a multi-scale feature adapter that converts heterogeneous multi-scale features with different spatial resolutions and channel dimensions into a unified representation. As a result, the same refinement architecture can be used across different base models without model-specific refiner redesign. On the multispectral stereo (MS2) dataset, the proposed method improves both BTS (big-to-small) and NeWCRFs (neural window fully connected CRFs), reducing the meter-based error metrics SqRel from 0.380 to 0.369 and RMSE from 3.163 to 3.126 for BTS, and reducing SqRel from 0.331 to 0.328 and RMSE from 2.937 to 2.924 for NeWCRFs. Qualitative results further show that the proposed method alleviates mixed-depth artifacts and abnormal depth patterns in regions lacking reliable depth supervision.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Vision and Imaging · Robotics and Sensor-Based Localization · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, Graduate School, Sangmyung University, Seoul 03016, Republic of Korea
Department of Computer Science, Sangmyung University, Seoul 03016, Republic of Korea
References (49)
Show source-provided references
1. Lavreniuk, M., and Lavreniuk, A. (2025, January 11–15). SPIdepth: Strengthened pose information for self-supervised monocular depth estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2025), Nashville, TN, USA.10.1109/CVPRW67362.2025.00091 ↗
2. SQLdepth: Generalizable self-supervised fine-structured monocular depth estimation
3. Bochkovskiy, A., Delaunoy, A., Germain, H., Santos, M., Zhou, Y., Richter, S.R., and Koltun, V. (2025, January 24–28). Depth Pro: Sharp monocular metric depth in less than a second. Proceedings of the Thirteenth International Conference on Learning Representations (ICLR 2025), Singapore.
4. Yang, L., Kang, B., Huang, Z., Xu, X., Feng, J., and Zhao, H. (2024, January 17–21). Depth Anything: Unleashing the power of large-scale unlabeled data. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024), Seattle, WA, USA.10.1109/CVPR52733.2024.00987 ↗
5. Masoumian, A., Rashwan, H.A., Cristiano, J., Asif, M.S., and Puig, D. (2022). Monocular Depth Estimation Using Deep Learning: A Review. Sensors, 22.10.3390/s22145353 ↗
6. Saunders, K., Vogiatzis, G., and Manso, L.J. (2023, January 2–6). Self-supervised monocular depth estimation: Let’s talk about the weather. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV 2023), Paris, France.10.1109/ICCV51070.2023.00818 ↗
7. Unsupervised monocular depth estimation for night-time images using adversarial domain feature adaptation10.1007/978-3-030-58604-1_27 ↗
8. Gasperini, S., Morbitzer, N., Jung, H., Navab, N., and Tombari, F. (2023, January 2–6). Robust monocular depth estimation under challenging conditions. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV 2023), Paris, France.10.1109/ICCV51070.2023.00751 ↗
9. Perception and sensing for autonomous vehicles under adverse weather conditions: A survey10.1016/j.isprsjprs.2022.12.021 ↗
10. Shin, U., Park, J., and Kweon, I.S. (2023, January 18–22). Deep Depth Estimation from Thermal Image. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00107 ↗
11. Shin, U., and Park, J. (2025). Deep depth estimation from thermal image: Dataset, benchmark, and challenges. arXiv.
12. Vision Meets Robotics: The KITTI Dataset10.1177/0278364913491297 ↗
13. Schilling, H., Gutsche, M., Brock, A., Spath, D., Rother, C., and Krispin, K. (2020, January 14–19). Mind the Gap—A benchmark for dense depth prediction beyond lidar. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2020), Virtual Event.10.1109/CVPRW50498.2020.00177 ↗
14. Non-Local Spatial Propagation Network for Depth Completion10.1007/978-3-030-58601-0_8 ↗
15. Guizilini, V., Ambrus, R., Pillai, S., Raventos, A., and Gaidon, A. (2020, January 14–19). 3D Packing for Self-Supervised Monocular Depth Estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), Virtual Event.10.1109/CVPR42600.2020.00256 ↗
16. Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., and Beijbom, O. (2020, January 14–19). nuScenes: A Multimodal Dataset for Autonomous Driving. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), Virtual Event.10.1109/CVPR42600.2020.01164 ↗
17. Kuznietsov, Y., Stückler, J., and Leibe, B. (2017, January 21–26). Semi-Supervised Deep Learning for Monocular Depth Map Prediction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, USA.10.1109/CVPR.2017.238 ↗
18. Gruber, T., Bijelic, M., Heide, F., Ritter, W., and Dietmayer, K. (2019, January 16–19). Pixel-Accurate Depth Evaluation in Realistic Driving Scenarios. Proceedings of the International Conference on 3D Vision (3DV 2019), Quebec City, QC, Canada.10.1109/3DV.2019.00020 ↗
19. Li, J., Wang, Y., Huang, Z., Zheng, J., Xian, K., Cao, Z., and Zhang, J. (November, January 29). Diffusion-Augmented Depth Prediction with Sparse Annotations. Proceedings of the 31st ACM International Conference on Multimedia (ACM MM 2023), Ottawa, ON, Canada.10.1145/3581783.3611807 ↗
20. Robust semi-supervised monocular depth estimation with reprojected distances
21. Singh, A.D., Ba, Y., Sarker, A., Zhang, H., Kadambi, A., Soatto, S., Srivastava, M., and Wong, A. (2023, January 18–22). Depth Estimation from Camera Image and mmWave Radar Point Cloud. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00895 ↗
22. Qin, Z., Xu, J., Zhao, W., Jiang, J., and Liu, X. (2024). Adaptive Stereo Depth Estimation with Multi-Spectral Images Across All Lighting Conditions. arXiv.
23. Xu, J., Liu, X., Jiang, J., Jiang, K., Li, R., Cheng, K., and Ji, X. (2025, January 19–23). Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging Scenarios. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2025), Atlanta, GA, USA.10.1109/ICRA55743.2025.11127354 ↗
24. Lu, Y., and Lu, G. (2021, January 5–9). An Alternative of LiDAR in Nighttime: Unsupervised Depth Estimation Based on Single Thermal Image. Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV 2021), Waikoloa, HI, USA.10.1109/WACV48630.2021.00388 ↗
25. Han, D., and Choi, Y. (2022). Monocular Depth Estimation from a Single Infrared Image. Electronics, 11.10.3390/electronics11111729 ↗
26. MonoTher-Depth: Enhancing Thermal Depth Estimation via Confidence-Aware Distillation10.1109/LRA.2025.3536855 ↗
27. Shin, U., Park, K., Lee, B.-U., Lee, K., and Kweon, I.S. (2023). Self-Supervised Monocular Depth Estimation from Thermal Images via Adversarial Multi-Spectral Adaptation. Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), IEEE.10.1109/WACV56688.2023.00575 ↗
28. External Multi-Modal Imaging Sensor Calibration for Sensor Fusion: A Review10.1016/j.inffus.2023.101806 ↗
29. Sensor Systems for Autonomous Vehicles: Functionality and Reliability Challenges in Adverse Environmental Conditions10.1016/j.measurement.2025.119215 ↗
30. Khan, M.A.U., Nazir, D., Pagani, A., Mokayed, H., Liwicki, M., Stricker, D., and Afzal, M.Z. (2022). A Comprehensive Survey of Depth Completion Approaches. Sensors, 22.10.20944/preprints202205.0343.v1 ↗
31. Learning depth with convolutional spatial propagation network10.1109/TPAMI.2019.2947374 ↗
32. CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion
33. Dynamic spatial propagation network for depth completion
34. Xu, Z., Yin, H., and Yao, J. (2020, January 25–28). Deformable spatial propagation networks for depth completion. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.10.1109/ICIP40778.2020.9191138 ↗
35. A Simple yet Universal Framework for Depth Completion
36. Jun, J., Lee, J.-H., and Kim, C.-S. (2024, January 17–21). Masked spatial propagation network for sparsity-adaptive depth refinement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.01869 ↗
37. Tang, J., Tian, F.-P., An, B., Li, J., and Tan, P. (2024, January 17–21). Bilateral propagation network for depth completion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.00932 ↗
38. Lee, J.H., Han, M.K., Ko, D.W., and Suh, I.H. (2019). From big to small: Multi-scale local planar guidance for monocular depth estimation. arXiv.
39. Yuan, W., Gu, X., Dai, Z., Zhu, S., and Tan, P. (2022, January 19–24). Neural window fully-connected CRFs for monocular depth estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00389 ↗
40. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11–17). Swin Transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Virtual Event.10.1109/ICCV48922.2021.00986 ↗
41. WTI-SLAM: A Novel Thermal Infrared Visual SLAM Algorithm for Weak Texture Thermal Infrared Images10.1007/s40747-025-01858-0 ↗
42. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer International Publishing.10.1007/978-3-319-24574-4_28 ↗
43. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Springer.10.1007/978-3-030-01234-2_49 ↗
44. Hariharan, B., Arbelaez, P., Girshick, R., and Malik, J. (2015). Hypercolumns for Object Segmentation and Fine-Grained Localization. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.10.1109/CVPR.2015.7298642 ↗
45. DetNet: Design Backbone for Object Detection10.1007/978-3-030-01240-3_21 ↗
46. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
47. Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature pyramid networks for object detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
48. Fu, H., Gong, M., Wang, C., Batmanghelich, K., and Tao, D. (2018, January 18–23). Deep ordinal regression network for monocular depth estimation. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA.10.1109/CVPR.2018.00214 ↗
49. Bhat, S.F., Alhashim, I., and Wonka, P. (2021, January 21–26). AdaBins: Depth estimation using adaptive bins. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021), Nashville, TN, USA.