Vehicle detection algorithm based on asymmetric fusion of radar point cloud and camera using deep learning

Aiming at the limitations of traditional target-level fusion methods for millimeter-wave radar and camera, such as low detection accuracy and poor performance in occluded or small object scenarios, this paper proposes a deep learning-based vehicle detection algorithm named Asy-RPCFNet. It asymmetrically fuses raw radar point clouds with image 2D detection boxes instead of radar detection boxes. This approach preserves the advantages of target-level fusion—efficient computation, simple architecture, and high robustness—while providing richer environmental information from point clouds to improve detection. The network incorporates prior knowledge including point cloud distribution consistency and distance-velocity consistency to better integrate both sensors. Evaluated on the nuScenes dataset, Asy-RPCFNet outperforms conventional fusion methods in accuracy under various conditions, improving pure visual detection by 2.7%~3.2% on sunny days and 2.5%~3.7% on rainy days, with comparable real-time performance. It effectively addresses visual detection weaknesses in occlusion, distant objects, and adverse weather.