MS3D: A Multi-Scale Feature Fusion 3D Object Detection Method for Autonomous Driving Applications
Source abstract
Abstract
With advancements in autonomous driving, LiDAR has become central to 3D object detection due to its precision and interference resistance. However, challenges such as point cloud sparsity and unstructured data persist. This study introduces MS3D (Multi-Scale Feature Fusion 3D Object Detection Method), a novel approach to 3D object detection that leverages the architecture of a 2D Convolutional Neural Network (CNN) as its core framework. It integrates a Second Feature Pyramid Network to enhance multi-scale feature representation and contextual integration. The Adam optimizer is employed for efficient adaptive parameter tuning, significantly improving detection performance. On the KITTI dataset, MS3D achieves average precisions of 93.58%, 90.91%, and 88.46% in easy, moderate, and hard scenarios, respectively, surpassing state-of-the-art models like VoxelNet, SECOND, and PointPillars.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Industrial Vision Systems and Defect Detection · Image Processing and 3D Reconstruction
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Chengyi College, Jimei University, Xiamen 361021, China
School of Ocean Information Engineering, Jimei University, Xiamen 361021, China
Funding
Natural Science Foundation of Fujian Province · Natural Science Foundation of Xiamen · Education and Scientific Research Project for Middle-Aged and Young Teachers of Fujian Province
References (45)
Show source-provided references
1. Zhou, Y., and Tuzel, O. (2018, January 18–22). VoxelNet: End-to-end learning for point cloud based 3D object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00472 ↗
2. Yan, Y., Mao, Y., and Li, B. (2018). SECOND: Sparsely embedded convolutional detection. Sensors, 18.10.3390/s18103337 ↗
3. Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., and Beijbom, O. (2019, January 16–20). PointPillars: Fast encoders for object detection from point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.01298 ↗
4. He, C., Zeng, H., Huang, J., Hua, X.-S., and Zhang, L. (2020, January 13–19). Structure aware single-stage 3D object detection from point cloud. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01189 ↗
5. Shi, S., Guo, C., Jiang, L., Wang, Z., Shi, J., Wang, X., and Li, H. (2020, January 13–19). Pv-rcnn: Point-voxel feature set abstraction for 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01054 ↗
6. Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., and Li, H. (2021, January 2–9). Voxel r-cnn: Towards high performance voxel-based 3D object detection. Proceedings of the AAAI Conference on Artificial Intelligence, Virtually.10.1609/aaai.v35i2.16207 ↗
7. Zhou, Y., Sun, P., Zhang, Y., Anguelov, D., Gao, J., Ouyang, T., Guo, J., Ngiam, J., and Vasudevan, V. (2020, January 16–18). End-to-end multi-view fusion for 3D object detection in lidar point clouds. Proceedings of the Conference on Robot Learning, PMLR, Virtual.
8. Liu, Z., Zhao, X., Huang, T., Hu, R., Zhou, Y., and Bai, X. (2020, January 7–12). TANet: Robust 3D object detection from point clouds with triple attention. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i07.6837 ↗
9. Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16–21). Are we ready for autonomous driving? The kitti vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.10.1109/CVPR.2012.6248074 ↗
10. Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., and Beijbom, O. (2020, January 13–19). nuscenes: A multimodal dataset for autonomous driving. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01164 ↗
11. Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., and Caine, B. (2020, January 13–19). Scalability in perception for autonomous driving: Waymo open dataset. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00252 ↗
12. Dong, Y., Kang, C., Zhang, J., Zhu, Z., Wang, Y., Yang, X., Su, H., Wei, X., and Zhu, J. (2023, January 17–24). Benchmarking robustness of 3D object detection to common corruptions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00105 ↗
13. Zheng, W., Tang, W., Chen, S., Jiang, L., and Fu, C.-W. (2021, January 2–9). Cia-ssd: Confident iou-aware single-stage object detector from point cloud. Proceedings of the AAAI Conference on Artificial Intelligence, Virtually.10.1609/aaai.v35i4.16470 ↗
14. Schuldt, C., Laptev, I., and Caputo, B. (2004, January 23–26). Recognizing human actions: A local SVM approach. Proceedings of the 17th International Conference on Pattern Recognition (ICPR 2004), Cambridge, UK.10.1109/ICPR.2004.1334462 ↗
15. Improved random forest for classification10.1109/TIP.2018.2834830 ↗
16. Yang, Z., Sun, Y., Liu, S., and Jia, J. (2020, January 13–19). 3Dssd: Point-based 3D single stage object detector. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01105 ↗
17. Zhang, Y., Hu, Q., Xu, G., Ma, Y., Wan, J., and Guo, Y. (2022, January 18–24). Not all points are equal: Learning highly efficient point-based detectors for 3D lidar point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01838 ↗
18. Wang, Y., Han, X., Wei, X., and Luo, J. (2024). Instance Segmentation Frustum–PointPillars: A Lightweight Fusion Algorithm for Camera–LiDAR Perception in Autonomous Driving. Mathematics, 12.10.3390/math12010153 ↗
19. Dong, P., Li, L., and Wei, Z. (2023, January 17–24). Diswot: Student architecture search for distillation without training. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01145 ↗
20. Chen, Y., Liu, J., Zhang, X., Qi, X., and Jia, J. (2023, January 17–24). Largekernel3D: Scaling up kernels in 3D sparse cnns. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01296 ↗
21. Zhou, C., Zhang, Y., Chen, J., and Huang, D. (2023, January 17–24). Octr: Octree-based transformer for 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00500 ↗
22. Chen, Y., Liu, J., Zhang, X., Qi, X., and Jia, J. (2023, January 17–24). Voxelnext: Fully sparse VoxelNet for 3D object detection and tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.02076 ↗
23. CasA: A cascade attention network for 3-D object detection from LiDAR point clouds10.1109/TGRS.2022.3203163 ↗
24. Xu, Q., Zhong, Y., and Neumann, U. (March, January 22). Behind the curtain: Learning occluded shapes for 3D object detection. Proceedings of the AAAI Conference on Artificial Intelligence, Virtually.
25. He, C., Li, R., Zhang, Y., Li, S., and Zhang, L. (2023, January 17–24). Msf: Motion-guided sequential fusion for efficient 3D object detection from point cloud sequences. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00503 ↗
26. Zhu, B., Wang, Z., Shi, S., Xu, H., Hong, L., and Li, H. (2023, January 17–24). Conquer: Query contrast voxel-detr for 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00897 ↗
27. Chen, X., Ma, H., Wan, J., Li, B., and Xia, T. (2017, January 21–26). Multi-view 3D object detection network for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.691 ↗
28. Huang, T., Liu, Z., Chen, X., and Bai, X. (2020, January 23–28). Epnet: Enhancing point features with image semantics for 3D object detection. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part XV 16.10.1007/978-3-030-58555-6_3 ↗
29. Vora, S., Lang, A.H., Helou, B., and Beijbom, O. (2020, January 13–19). Pointpainting: Sequential fusion for 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00466 ↗
30. Yin, T., Zhou, X., and Krähenbühl, P. (2021, January 6–14). Multimodal virtual point 3D detection. Proceedings of the 35th International Conference on Neural Information Processing Systems, Online.
31. Li, X., Ma, T., Hou, Y., Shi, B., Yang, Y., Liu, Y., Wu, X., Chen, Q., Li, Y., and Qiao, Y. (2023, January 17–24). Logonet: Towards accurate 3D object detection with local-to-global cross-modal fusion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01681 ↗
32. Wu, X., Peng, L., Yang, H., Xie, L., Huang, C., Deng, C., Liu, H., and Cai, D. (2022, January 18–24). Sparse fuse dense: Towards high quality 3D detection with depth completion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00534 ↗
33. Wu, H., Wen, C., Shi, S., Li, X., and Wang, C. (2023, January 17–24). Virtual sparse convolution for multimodal 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.02074 ↗
34. Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21–26). Pointnet: Deep learning on point sets for 3D classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.
35. Qi, C.R., Yi, L., Su, H., and Guibas, L.J. (2017, January 4–9). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
36. Kingma, D.P. (2014). Adam: A method for stochastic optimization. arXiv.
37. Girshick, R. (2015, January 7–13). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
38. An algorithm for visual simultaneous localization and mapping with integrated hybrid attention instance segmentation
39. Wang, T., Hu, X., Liu, Z., and Fu, C.-W. (December, January 28). Sparse2Dense: Learning to densify 3D features for 3D object detection. Proceedings of the 36th International Conference on Neural Information Processing Systems, New Orleans, LA, USA.
40. Wu, X., Tian, Z., Wen, X., Peng, B., Liu, X., Yu, K., and Zhao, H. (2024, January 16–22). Towards large-scale 3D representation learning with multi-dataset point prompt training. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.01849 ↗
41. Liu, Y., Kong, L., Wu, X., Chen, R., Li, X., Pan, L., Liu, Z., and Ma, Y. (2024, January 16–22). Multi-Space Alignments Towards Universal LiDAR Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.01388 ↗
42. Jiao, Y., Jie, Z., Chen, S., Chen, J., Ma, L., and Jiang, Y.-G. (2023, January 17–24). Msmdfusion: Fusing lidar and camera at multiple scales with multi-depth seeds for 3D object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.02073 ↗
43. Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., and Liao, K.-D. (2024, January 3–8). A survey on multimodal large language models for autonomous driving. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACVW60836.2024.00106 ↗
44. Uni-to-Multi Modal Knowledge Distillation for Bidirectional LiDAR-Camera Semantic Segmentation10.1109/TPAMI.2024.3451658 ↗
45. Point-to-pixel prompting for point cloud analysis with pre-trained image models10.1109/TPAMI.2024.3354961 ↗