Focusing 3D Small Objects with Object Matching Set Abstraction
Source abstract
Abstract
Currently, 3D object detection methods fail to detect small objects due to the fewer effective points of small objects. It is a significant challenge to reduce the loss of information of points in representation learning. To this end, we propose an effective 3D detection method with object matching set abstraction (OMSA). We observe that key points are lost during feature learning with multiple set abstraction layers, especially for downsampling and queries. Therefore, we present a novel sampling module named focus-based sampling, which raises the sampling probability of small objects. In addition, we design a multi-scale cube query to match the small objects with a close geometric alignment. Our comprehensive experimental evaluations on the KITTI 3D benchmark demonstrate significant performance improvements in 3D object detection. Notably, the proposed framework exhibits competitive detection accuracy for small objects (pedestrians and cyclists). Through an ablation study, we verify that each module contributes to the performance enhancement and demonstrate the robustness of the method against the balance factor.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotics and Sensor-Based Localization · Image Processing and 3D Reconstruction · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Shanxi Key Laboratory of Machine Vision and Virtual Reality, North University of China, Taiyuan 030051, China
School of Computer Science and Technology, North University of China, Taiyuan 030051, China
Funding
Natural Science Foundation of Shanxi Province · Foundation of Shanxi Key Laboratory of Machine Vision and Virtual Reality
References (44)
Show source-provided references
1. 3D object detection for autonomous driving: A comprehensive survey10.1007/s11263-023-01790-1 ↗
2. Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21–26). Pointnet: Deep learning on point sets for 3D classification and segmentation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.
3. Survey and systematization of 3D object detection models and methods10.1007/s00371-023-02891-1 ↗
4. Zhou, Y., and Tuzel, O. (2018, January 18–22). Voxelnet: End-to-end learning for point cloud based 3D object detection. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00472 ↗
5. Yan, Y., Mao, Y., and Li, B. (2018). Second: Sparsely embedded convolutional detection. Sensors, 18.10.3390/s18103337 ↗
6. VP-net: Voxels as points for 3D object detection
7. Chen, Y., Yu, Z., Chen, Y., Lan, S., Anandkumar, A., Jia, J., and Alvarez, J.M. (2023, January 1–6). Focalformer3D: Focusing on hard instance for 3D object detection. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.00771 ↗
8. Yin, T., Zhou, X., and Krahenbuhl, P. (2021, January 20–25). Center-based 3D object detection and tracking. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01161 ↗
9. Chi, X., Liu, J., Lu, M., Zhang, R., Wang, Z., Guo, Y., and Zhang, S. (2023, January 17–24). BEV-SAN: Accurate bev 3D object detection via slice attention networks. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.01675 ↗
10. Wang, Y., Deng, J., Hou, Y., Li, Y., Zhang, Y., Ji, J., Ouyang, W., and Zhang, Y. (2024). CluB: Cluster Meets BEV for LiDAR-Based 3D Object Detection. Advances in Neural Information Processing Systems, MIT Press.
11. Li, Z., Lan, S., Alvarez, J.M., and Wu, Z. (2024, January 16–22). BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.01901 ↗
12. Li, J., Luo, C., and Yang, X. (2023, January 17–24). PillarNeXt: Rethinking network designs for 3D object detection in LiDAR point clouds. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.01685 ↗
13. Qi, C.R., Yi, L., Su, H., and Guibas, L.J. (2017). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in Neural Information Processing Systems, MIT Press.
14. Yang, Z., Sun, Y., Liu, S., and Jia, J. (2020, January 13–19). 3DSSD: Point-based 3D single stage object detector. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01105 ↗
15. Super sparse 3D object detection10.1109/TPAMI.2023.3286409 ↗
16. Towards large-scale small object detection: Survey and benchmarks
17. A full data augmentation pipeline for small object detection based on generative adversarial networks10.1016/j.patcog.2022.108998 ↗
18. Boosting R-CNN: Reweighting R-CNN samples by RPN’s error for underwater object detection10.1016/j.neucom.2023.01.088 ↗
19. Context Feature Integration and Balanced Sampling Strategy for Small Weak Object Detection in Remote Sensing Imagery
20. Context-aware block net for small object detection10.1109/TCYB.2020.3004636 ↗
21. Global to Local: A Scale-Aware Network for Remote Sensing Object Detection10.1109/TGRS.2023.3294241 ↗
22. DsP-YOLO: An anchor-free network with DsPAN for small object detection of multiscale defects10.1016/j.eswa.2023.122669 ↗
23. Real-Time Object Detection Network in UAV-Vision Based on CNN and Transformer10.1109/TIM.2023.3241825 ↗
24. UIU-Net: U-Net in U-Net for infrared small object detection10.1109/TIP.2022.3228497 ↗
25. Tang, S., Zhang, S., and Fang, Y. (2024, January 13–17). HIC-YOLOv5: Improved YOLOv5 For Small Object Detection. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan.10.1109/ICRA57147.2024.10610273 ↗
26. Chen, C., Chen, Z., Zhang, J., and Tao, D. (March, January 22). SASA: Semantics-augmented set abstraction for point-based 3D object detection. Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI-22), Online.10.1609/aaai.v36i1.19897 ↗
27. Density-Aware and Semantic-Guided Fusion for 3D Object Detection using LiDAR-Camera Sensors10.1109/JSEN.2023.3302314 ↗
28. PSA-Det3D: Pillar set abstraction for 3D object detection10.1016/j.patrec.2023.03.016 ↗
29. WS-SSD: Achieving faster 3D object detection for autonomous driving via weighted point cloud sampling10.1016/j.eswa.2024.123805 ↗
30. Huang, Z., Wang, Y., Tang, X., and Sun, H. (2023, January 18–23). Boundary-aware set abstraction for 3D object detection. Proceedings of the 2023 International Joint Conference on Neural Networks (IJCNN), Gold Coast, Australia.10.1109/IJCNN54540.2023.10191728 ↗
31. PV-RCNN++: Point-voxel feature set abstraction with local vector representation for 3D object detection10.1007/s11263-022-01710-9 ↗
32. Zhang, Y., Huang, D., Wang, Y., Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16–21). Are we ready for autonomous driving? the kitti vision benchmark suite. Proceedings of the 2012 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, .
33. Shi, S., Wang, X., and Li, H. (2019, January 15–20). PointRCNN: 3D object proposal generation and detection from point cloud. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00086 ↗
34. Shi, S., Guo, C., Jiang, L., Wang, Z., Shi, J., Wang, X., and Li, H. (2020, January 13–19). PV-RCNN: Point-voxel feature set abstraction for 3D object detection. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01054 ↗
35. Jiang, T., Song, N., Liu, H., Yin, R., Gong, Y., and Yao, J. (June, January 30). VIC-Net: Voxelization information compensation network for point cloud 3D object detection. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, China.10.1109/ICRA48506.2021.9561597 ↗
36. ASCNet: 3D object detection from point cloud based on adaptive spatial context features10.1016/j.neucom.2021.12.061 ↗
37. Zhang, Y., Hu, Q., Xu, G., Ma, Y., Wan, J., and Guo, Y. (2022, January 18–24). Not all points are equal: Learning highly efficient point-based detectors for 3D lidar point clouds. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01838 ↗
38. Yang, Z., Sun, Y., Liu, S., Shen, X., and Jia, J. (2018). IPOD: Intensive point-based object detector for point cloud. arXiv.10.1109/ICCV.2019.00204 ↗
39. Ku, J., Mozifian, M., Lee, J., Harakeh, A., and Waslander, S.L. (2018, January 1–5). Joint 3D proposal generation and object detection from view aggregation. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8594049 ↗
40. Qi, C.R., Liu, W., Wu, C., Su, H., and Guibas, L.J. (2018, January 18–23). Frustum pointnets for 3D object detection from RGB-D data. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00102 ↗
41. Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., and Beijbom, O. (2019, January 15–20). PointPillars: Fast encoders for object detection from point clouds. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01298 ↗
42. Wang, Z., and Jia, K. (2019, January 3–8). Frustum ConvNet: Sliding frustums to aggregate local point-wise features for amodal 3D object detection. Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.10.1109/IROS40897.2019.8968513 ↗
43. Zhou, D., Fang, J., Song, X., Liu, L., Yin, J., Dai, Y., Li, H., and Yang, R. (2020, January 13–19). Joint 3D instance segmentation and object detection for autonomous driving. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00191 ↗
44. Zhang, Y., Huang, D., and Wang, Y. (2021, January 2–9). PC-RGNN: Point cloud completion and graph neural network for 3D object detection. Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI-21), Online.10.1609/aaai.v35i4.16456 ↗