Research Progress on Multi-Modal Fusion Object Detection Algorithms for Autonomous Driving: A Review
Source abstract
Abstract
As the number and complexity of sensors in autonomous vehicles continue to rise, multimodal fusion-based object detection algorithms are increasingly being used to detect 3D environmental information, significantly advancing ... | Find, read and cite all the research you need on Tech Science Press
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Marine and Coastal Research · Advanced Technology in Applications · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (100+)
Show source-provided references
1. Explainable artificial intelligence for autonomous driving: a comprehensive overview and field guide for future research directions10.1109/ACCESS.2024.3431437 ↗
2. Multi-modal 3D object detection in autonomous driving: a survey10.1007/s11263-023-01784-z ↗
3. TSF: two-stage sequential fusion for 3D object detection10.1109/JSEN.2022.3175192 ↗
4. A review of object detection based on convolutional neural network
5. Transformer for object detection: review and benchmark10.1016/j.engappai.2023.107021 ↗
6. Object detection with deep learning: a review10.1109/TNNLS.2018.2876865 ↗
7. Are we ready for autonomous driving? The KITTI vision benchmark suite
8. nuScenes: a multimodal dataset for autonomous driving
9. Scalability in perception for autonomous driving: waymo open dataset
10. Mao J, Niu M, Jiang C, Liang H, Chen J, Liang X, et al. One million scenes for autonomous driving: once dataset. arXiv:2106.11037. 2021.
11. PersFormer: 3D lane detection via perspective transformer and the OpenLane benchmark
12. Wang · A motion and accident prediction benchmark for V2X autonomous driving · 2024 · 5599
13. The ApolloScape open dataset for autonomous driving and its application10.1109/TPAMI.2019.2926463 ↗
14. Houston J, Zuidhof G, Bergamini L, Ye Y, Chen L, Jain A, et al. One thousand and one hours: self-driving motion prediction dataset. arXiv:2006.14480. 2020.
15. A*3D dataset: towards autonomous driving in challenging environments
16. The H3D dataset for full-surround 3D multi-object detection and tracking in crowded urban scenes
17. Geyer J, Kassahun Y, Mahmudi M, Ricou X, Durgesh R, Chung AS, et al. A2D2: audi autonomous driving dataset. arXiv:2004.06320. 2020.
18. Gählert N, Jourdan N, Cordts M, Franke U, Denzler J. Cityscapes 3D: dataset and benchmark for 9 DoF vehicle detection. arXiv:2006.07864. 2020.
19. Argoverse: 3D tracking and forecasting with rich maps
20. Weng X, Man Y, Cheng D, Yuan Y, O’Toole M, Kitani K. All-in-one drive: a large-scale comprehensive perception dataset with high-density long-range point clouds; 2020. doi:10.13140/RG.2.2.21621.81122.
21. KITTI-360: a novel dataset and benchmarks for urban scene understanding in 2D and 3D10.1109/TPAMI.2022.3179507 ↗
22. CenterNet: keypoint triplets for object detection
23. M3D-RPN: monocular 3D region proposal network for object detection
24. ROI-10D: monocular lifting of 2D detection to 6D pose and metric shape
25. Face detection using deep learning: an improved faster RCNN approach10.1016/j.neucom.2018.03.030 ↗
26. Monocular 3D object detection for autonomous driving
27. Deep MANTA: a coarse-to-fine many-task network for joint 2D and 3D vehicle analysis from monocular image
28. Mono3D++: monocular 3D vehicle detection with two-scale 3D hypotheses and task priors
29. 3D-RCNN: instance-level 3D object reconstruction via render-and-compare
30. Disp R-CNN: stereo 3D object detection via shape prior guided instance disparity estimation
31. Stereo R-CNN based 3D object detection for autonomous driving
32. Triangulation learning network: from monocular to stereo 3D object detection
33. Lin X, Pei Z, Lin T, Huang L, Su Z. Sparse4D v3: advancing end-to-end 3D detection and tracking. arXiv:2311.11722. 2023.
34. Far3D: expanding the horizon for surround-view 3D object detection
35. Li B, Zhang T, Xia T. Vehicle detection from 3D lidar using fully convolutional network. arXiv:1608.07916. 2016.
36. BirdNet: a 3D object detection framework from LiDAR information
37. BirdNet+: end-to-end 3D object detection in LiDAR bird’s eye view
38. End-to-end multi-view fusion for 3D object detection in LiDAR point clouds
39. FocalFormer3D: focusing on hard instance for 3D object detection
40. VoxelNet: end-to-end learning for point cloud based 3D object detection
41. Structure aware single-stage 3D object detection from point cloud
42. Fully sparse transformer 3-D detector for LiDAR point cloud10.1109/TGRS.2023.3328929 ↗
43. PointRCNN: 3d object proposal generation and detection from point cloud
44. Qi CR, Yi L, Su H, Guibas LJ. PointNet++: deep hierarchical feature learning on point sets in a metric space. arXiv:1706.02413. 2017.
45. 3DSSD: point-based 3D single stage object detector
46. PointPillars: fast encoders for object detection from point clouds
47. PointNet: deep learning on point sets for 3D classification and segmentation
48. Pillar-based object detection for autonomous driving
49. Point-GNN: graph neural network for 3D object detection in a point cloud
50. SVGA-net: sparse voxel-graph attention network for 3D object detection from point clouds
51. SAT-GCN: self-attention graph convolutional network-based 3D object detection for autonomous driving10.1016/j.knosys.2022.110080 ↗
52. Shift R-CNN: deep monocular 3D object detection with closed-form geometric constraints
53. Pseudo- LiDAR++: accurate depth for 3D object detection in autonomous driving
54. Confidence guided stereo 3D object detection with split depth estimation
55. Frustum PointNets for 3D object detection from RGB-D data
56. Frustum ConvNet: sliding Frustums to aggregate local point-wise features for amodal 3D object detection
57. Frustum-PointPillars: a multi-stage approach for 3D object detection using RGB camera and LiDAR
58. Multi-view 3D object detection network for autonomous driving
59. PI-RCNN: an efficient multi-sensor 3D object detector with point-based attentive cont-conv fusion module
60. Scanet: spatial-channel attention network for 3D object detection
61. FusionPainting: multimodal fusion with adaptive attention for 3D object detection
62. Graph R-CNN for scene graph generation
63. FusionRCNN: lidar-camera fusion for two-stage 3D object detection10.3390/rs15071839 ↗
64. TransFusion: robust LiDAR-camera fusion for 3D object detection with transformers
65. Vaswanic · 2017 · 6000
66. CLOCs: camera-LiDAR object candidates fusion for 3D object detection
67. Fast-CLOCs: fast camera-LiDAR object candidates fusion for 3D object detection
68. Multi-view frustum pointnet for object detection in autonomous driving
69. ConCs-fusion: a context clustering-based radar and camera fusion for three-dimensional object detection10.3390/rs15215130 ↗
70. Joint 3D proposal generation and object detection from view aggregation
71. A multi-sensor fusion based 2D-driven 3D object detection approach for large scene applications
72. Chen Z, Li Z, Zhang S, Fang L, Jiang Q, Zhao F, et al. AutoAlign: pixel-instance feature aggregation for multi-modal 3D object detection. arXiv:2201.06493. 2022.10.24963/ijcai.2022/116 ↗
73. Chen Z, Li Z, Zhang S, Fang L, Jiang Q, Zhao F. AutoAlignV2: deformable feature aggregation for dynamic multi-modal 3D object detection. arXiv:2207.10316. 2022.10.1007/978-3-031-20074-8_36 ↗
74. Emerging trends in autonomous vehicle perception: multimodal fusion for 3D object detection10.3390/wevj15010020 ↗
75. Multi-sem fusion: multimodal semantic fusion for 3-D object detection
76. Unifying voxel-based representation with transformer for 3d object detection
77. Song Z, Zhang G, Xie J, Liu L, Jia C, Xu S, et al. VoxelNextFusion: a simple, unified and effective voxel fusion framework for multi-modal 3D object detection. arXiv:2401.02702. 2024.10.1109/TGRS.2023.3331893 ↗
78. BAFusion: bidirectional attention fusion for 3D object detection based on LiDAR and camera10.3390/s24144718 ↗
79. Deng J, Zhang S, Dayoub F, Ouyang W, Zhang Y, Reid I. PoIFusion: multi-modal 3D object detection via fusion at points of interest. arXiv:2403.09212. 2024.
80. VoPiFNet: voxel-pixel fusion network for multi-class 3D object detection10.1109/TITS.2024.3392783 ↗
81. Multi-layer fusion 3D object detection via lidar point cloud and camera image10.3390/app14041348 ↗
82. Zhu X, Su W, Lu L, Li B, Wang X, Dai J, et al. Deformable DETR: deformable transformers for end-to-end object detection. arXiv:2010.04159. 2020.
83. Cross modal transformer: towards fast and robust 3D object detection
84. CAT-det: contrastively augmented transformer for multimodal 3D object detection
85. Camera-radar fusion with modality interaction and radar Gaussian expansion for 3D object detection10.34133/cbsystems.0079 ↗
86. Sparse fuse dense: towards high quality 3D detection with depth completion
87. Robust detection of non-motorized road users using deep learning on optical and LIDAR data
88. Faster R-CNN: towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
89. Huang · Computer vision–ECCV 2020 · 2020 · 3510.1007/978-3-030-58555-6_3 ↗
90. LIGA-stereo: learning LiDAR geometry aware representations for stereo-based 3D detector
91. Snow-CLOCs: camera-LiDAR object candidate fusion for 3D object detection in snowy conditions10.3390/s24134158 ↗
92. InceptionNeXt: when inception meets ConvNeXt
93. Jocher · ultralytics/yolov5: v7. 0-yolov5 sota realtime instance segmentation · 2022
94. SECOND: sparsely embedded convolutional detection10.3390/s18103337 ↗
95. BEVFusion: multi-task multi-sensor fusion with unified bird’s-eye view representation
96. IS-fusion: instance-scene collaborative fusion for multimodal 3D object detection
97. GAFusion: adaptive fusing LiDAR and camera with multiple guidance for 3D object detection
98. Zhao H, Guan R, Wu T, Man KL, Yu L, Yue Y. UniBEVFusion: unified radar-vision BEVFusion for 3D object detection. arXiv:2409.14751. 2024.10.1109/ICRA55743.2025.11128067 ↗
99. RCM-fusion: radar-camera multi-level fusion for 3D object detection
100. BEV-radar: bidirectional radar-camera fusion for 3D object detection10.52396/JUSTC-2023-0006 ↗