UDF-3D: Uncertainty-Driven Decision-Level Fusion for Camera–LiDAR 3D Object Detection
Source abstract
Abstract
Camera and LiDAR provide highly complementary information, and effective fusion of both modalities is desirable for 3D object detection. However, existing decision-level fusion methods mainly rely on the confidence of objects while neglecting the object uncertainty. To address this, we propose UDF-3D, an uncertainty-driven camera–LiDAR decision-level fusion method based on Dempster–Shafer evidence theory. First, object uncertainty is quantified by introducing the theory of subjective logic, where subjective opinions incorporate category belief masses and an uncertainty mass. Second, a cost matrix is designed for object matching, where each element is a weighted combination of geometric and semantic information from both sensors, and the weights are determined by the uncertainty parameters. Third, we construct a view-frustum constraint to re-evaluate unmatched objects, thereby reducing the false-negative rate. Finally, we design a novel evidence discounting factor within the Dempster–Shafer framework for matched objects, thereby mitigating cross-modal object conflicts during fusion and improving detection accuracy. Experiments on the KITTI dataset demonstrate that the proposed method outperforms existing decision-level fusion approaches, yielding improved detection accuracy.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Advanced Image Fusion Techniques · Visual Attention and Saliency Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electrical and Control Engineering, Shaanxi University of Science and Technology, Xi’an 710021, China
School of Mechanical and Precision Instrument Engineering, Xi’an University of Technology, Xi’an 710048, China
Funding
Natural Science Foundation of China
References (38)
Show source-provided references
1. Multi-modal 3D object detection in autonomous driving: A survey and taxonomy10.1109/TIV.2023.3264658 ↗
2. Qian, R., Lai, X., and Li, X. (2021). 3D object detection for autonomous driving: A survey. arXiv.10.1016/j.patcog.2022.108796 ↗
3. Yan, Y., Mao, Y., and Li, B. (2018). SECOND: Sparsely embedded convolutional detection. Sensors, 18.10.3390/s18103337 ↗
4. Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., and Beijbom, O. (2019). PointPillars: Fast encoders for object detection from point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 15–20 June 2019, IEEE.10.1109/CVPR.2019.01298 ↗
5. From points to parts: 3D object detection from point cloud with part-aware and part-aggregation network
6. Shi, S., Wang, X., and Li, H. (2019). PointRCNN: 3D object proposal generation and detection from point cloud. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 15–20 June 2019, IEEE.10.1109/CVPR.2019.00086 ↗
7. Shi, S., Guo, C., Jiang, L., Wang, Z., Shi, J., Wang, X., and Li, H. (2020). PV-RCNN: Point-voxel feature set abstraction for 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020, IEEE.10.1109/CVPR42600.2020.01054 ↗
8. Yin, T., Zhou, X., and Krähenbühl, P. (2021). Center-based 3D object detection and tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.10.1109/CVPR46437.2021.01161 ↗
9. Joint semantic segmentation using representations of LiDAR point clouds and camera images10.1016/j.inffus.2024.102370 ↗
10. Yu, K., Tao, T., Xie, H., Lin, Z., Liang, T., Wang, B., Chen, P., Hao, D., Wang, Y., and Liang, X. (2023). Benchmarking the robustness of LiDAR-camera fusion for 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada, 17–24 June 2023, IEEE.10.1109/CVPRW59228.2023.00321 ↗
11. The combination of evidence in the transferable belief model10.1109/34.55104 ↗
12. Xu, D., Anguelov, D., and Jain, A. (2018). PointFusion: Deep sensor fusion for 3D bounding box estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.10.1109/CVPR.2018.00033 ↗
13. Vora, S., Lang, A.H., Helou, B., and Beijbom, O. (2020). PointPainting: Sequential fusion for 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 16–18 June 2020, IEEE.10.1109/CVPR42600.2020.00466 ↗
14. Fast and accurate 3D object detection for LiDAR-camera-based autonomous vehicles using one shared voxel-based backbone10.1109/ACCESS.2021.3055491 ↗
15. Wang, C., Ma, C., Zhu, M., and Yang, X. (2021). PointAugmenting: Cross-modal augmentation for 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 20–25 June 2021, IEEE.10.1109/CVPR46437.2021.01162 ↗
16. Xu, S., Zhou, D., Fang, J., Yin, J., Bin, Z., and Zhang, L. (2021). FusionPainting: Multimodal fusion with adaptive attention for 3D object detection. Proceedings of the IEEE Intelligent Transportation Systems Conference (ITSC), Indianapolis, IN, USA, 19–22 September 2021, IEEE.10.1109/ITSC48978.2021.9564951 ↗
17. Chen, X., Ma, H., Wan, J., Li, B., and Xia, T. (2017). Multi-view 3D object detection network for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.10.1109/CVPR.2017.691 ↗
18. EPNet++: Cascade bi-directional fusion for multi-modal 3D object detection
19. Li, Y., Yu, A.W., Meng, T., Caine, B., Ngiam, J., Peng, D., Shen, J., Lu, Y., Zhou, D., and Le, Q.V. (2022). DeepFusion: LiDAR-camera deep fusion for multi-modal 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022, IEEE.10.1109/CVPR52688.2022.01667 ↗
20. Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D.L., and Han, S. (2023). BEVFusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), London, UK, 29 May–2 June 2023, IEEE.10.1109/ICRA48891.2023.10160968 ↗
21. LiDAR-camera fusion: Dual transformer enhancement for 3D object detection10.1016/j.engappai.2022.105815 ↗
22. Zhang, Y., Liu, K., Bao, H., Qian, X., Wang, Z., Ye, S., and Wang, W. (2023). AFTR: A robustness multi-sensor fusion model for 3D object detection based on adaptive fusion transformer. Sensors, 23.10.3390/s23208400 ↗
23. UncertainBEV: Uncertainty-aware BEV fusion for roadside 3D object detection10.1016/j.imavis.2025.105567 ↗
24. Pang, S., Morris, D., and Radha, H. (2020). CLOCs: Camera-LiDAR object candidates fusion for 3D object detection. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 24 October 2020, IEEE.10.1109/IROS45743.2020.9341791 ↗
25. Pang, S., Morris, D., and Radha, H. (2022). Fast-CLOCs: Fast camera-LiDAR object candidates fusion for 3D object detection. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 3–8 January 2022, IEEE.10.1109/WACV51458.2022.00380 ↗
26. Shen, B., Dai, S., Chen, Y., Xiong, R., Wang, Y., and Jiao, Y. (2023). GOOD: General optimization-based fusion for 3D object detection via LiDAR-camera object candidates. arXiv.10.2139/ssrn.4992265 ↗
27. Sensoy, M., Kaplan, L., and Kandemir, M. (2018). Evidential deep learning to quantify classification uncertainty. Advances in Neural Information Processing Systems (NeurIPS), Curran Associates, Inc.
28. Nallapareddy, M.R., Sirohi, K., Drews, P.L.J., Burgard, W., Cheng, C.-H., and Valada, A. (2023). EvCenterNet: Uncertainty estimation for object detection using evidential learning. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA, 1–5 October 2023, IEEE.10.1109/IROS55552.2023.10341826 ↗
29. Jøsang, A. (2016). Subjective Logic: A Formalism for Reasoning Under Uncertainty, Springer.10.1007/978-3-319-42337-1 ↗
30. Upper and lower probabilities induced by a multivalued mapping10.1214/aoms/1177698950 ↗
31. Belief functions contextual discounting and canonical decompositions10.1016/j.ijar.2011.06.005 ↗
32. Geiger, A., Lenz, P., and Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, 16–21 June 2012, IEEE.10.1109/CVPR.2012.6248074 ↗
33. Varghese, R., and Sambath, M. (2024). YOLOv8: A novel object detection algorithm with enhanced performance and robustness. Proceedings of the International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS), Chennai, India, 17–18 April 2024, IEEE.10.1109/ADICS58448.2024.10533619 ↗
34. Zhao, Y., Lv, W., Xu, S., Wei, J., Wang, G., Dang, Q., Liu, Y., and Chen, J. (2024). DETRs beat YOLOs on real-time object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 16–22 June 2024, IEEE.10.1109/CVPR52733.2024.01605 ↗
35. Qi, C.R., Liu, W., Wu, C., Su, H., and Guibas, L.J. (2018). Frustum PointNets for 3D object detection from RGB-D data. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018, IEEE.10.1109/CVPR.2018.00102 ↗
36. Paigwar, A., Sierra-Gonzalez, D., Erkent, Ö., and Laugier, C. (2021). Frustum-PointPillars: A multi-stage approach for 3D object detection using RGB camera and LiDAR. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada, 11–17 October 2021, IEEE.10.1109/ICCVW54120.2021.00327 ↗
37. Yoo, J.H., Kim, Y., Kim, J., and Choi, J.W. (2020). 3D-CVF: Generating joint camera and LiDAR features using cross-view spatial feature fusion for 3D object detection. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK, 23–28 August 2020, IEEE.10.1007/978-3-030-58583-9_43 ↗
38. Zhang, Y., Chen, J., and Huang, D. (2022). CAT-Det: Contrastively augmented transformer for multi-modal 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022, IEEE.10.1109/CVPR52688.2022.00098 ↗