AuxDepthNet: Real-Time Monocular 3D Object Detection with Depth-Sensitive Features
Source abstract
Abstract
Monocular 3D object detection is a challenging task in autonomous systems due to the lack of explicit depth information in single-view images. Existing methods often depend on external depth estimators or expensive sensors, which increase computational complexity and complicate integration into existing systems. To overcome these limitations, we propose AuxDepthNet, an efficient framework for real-time monocular 3D object detection that eliminates the reliance on external depth maps or pre-trained depth models. AuxDepthNet introduces two key components: the Auxiliary Depth Feature (ADF) module, which implicitly learns depth-sensitive features to improve spatial reasoning and computational efficiency, and the Depth Position Mapping (DPM) module, which embeds depth positional information directly into the detection process to enable accurate object localization and 3D bounding box regression. Leveraging the DepthFusion Transformer (DFT) architecture, AuxDepthNet globally integrates visual and depth-sensitive features through depth-guided interactions, ensuring robust and efficient detection. Extensive experiments on the KITTI dataset show that AuxDepthNet achieves state-of-the-art performance, with AP3D scores of 24.72% (Easy), 18.63% (Moderate), and 15.31% (Hard), and APBEV scores of 34.11% (Easy), 25.18% (Moderate), and 21.90% (Hard) at an IoU threshold of 0.7.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Advanced Vision and Imaging · Advanced Image and Video Retrieval Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Mechanical Engineering, National Korea Maritime and Ocean University, Busan 49112, Republic of Korea
Maritime ICT and Mobility Research Department, Korea Institute of Ocean Science and Technology, Busan 49111, Republic of Korea
Funding
Korea Institute of Marine Science & Technology Promotion · Ministry of Science and ICT, the Republic of Korea
References (49)
Show source-provided references
1. He, C., Zeng, H., Huang, J., Hua, X.-S., and Zhang, L. (2020, January 13–19). Structure aware single-stage 3d object detection from point cloud. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01189 ↗
2. Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., and Beijbom, O. (2019, January 15–20). Pointpillars: Fast encoders for object detection from point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.01298 ↗
3. Shi, S., Wang, X., and Li, H. (2019, January 15–20). Pointrcnn: 3d object proposal generation and detection from point cloud. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00086 ↗
4. Li, P., Chen, X., and Shen, S. (2019, January 15–20). Stereo r-cnn based 3d object detection for autonomous driving. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00783 ↗
5. Wang, Y., Chao, W.-L., Garg, D., Hariharan, B., Campbell, M., and Weinberger, K.Q. (2019, January 15–20). Pseudo-lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00864 ↗
6. Brazil, G., Pons-Moll, G., Liu, X., and Schiele, B. (2020). Kinematic 3d object detection in monocular video. Computer Vision–ECCV 2020, Proceedings of the 16th European Conference, Glasgow, UK, 23–28 August 2020, Proceedings, Part XXIII 16, Springer International Publishing.10.1007/978-3-030-58592-1_9 ↗
7. Chen, Y., Tai, L., Sun, K., and Li, M. (2020, January 13–19). Monopair: Monocular 3d object detection using pairwise spatial relationships. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01211 ↗
8. Ku, J., Pon, A.D., and Waslander, S.L. (2019, January 15–20). Monocular 3d object detection leveraging accurate proposals and shape reconstruction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.01214 ↗
9. Git: Graph interactive transformer for vehicle re-identification10.1109/TIP.2023.3238642 ↗
10. Shen, F., Shu, X., Du, X., and Tang, J. (2023, January 27). Pedestrian-specific bipartite-aware similarity learning for text-based person retrieval. Proceedings of the 31st ACM International Conference on Multimedia, New York, NY, USA.10.1145/3581783.3612009 ↗
11. Ma, X., Wang, Z., Li, H., Zhang, P., Ouyang, W., and Fan, X. (2019, January 27–28). Accurate monocular 3d object detection via color-embedded 3d reconstruction for autonomous driving. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.10.1109/ICCV.2019.00695 ↗
12. Weng, X., and Kitani, K. (2019, January 27–28). Monocular 3d object detection with pseudo-lidar point cloud. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea.10.1109/ICCVW.2019.00114 ↗
13. Ding, M., Huo, Y., Yi, H., Wang, Z., Shi, J., Lu, Z., and Luo, P. (2020, January 13–19). Learning depth-guided convolutions for monocular 3d object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00508 ↗
14. Ouyang, E., Zhang, L., Chen, M., Arnab, A., and Fu, Y. (December, January 30). Dynamic depth fusion and transformation for monocular 3d object detection. Proceedings of the Asian Conference on Computer Vision, Kyoto, Japan.
15. Kim, B., Lee, J., Kang, J., Kim, E.-S., and Kim, H.J. (2021, January 20–25). Hotr: End-to-end human-object interaction detection with transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00014 ↗
16. Zhu, X., Su, W., Lu, L., Li, B., Wang, X., and Dai, J. (2020). Deformable detr: Deformable transformers for end-to-end object detection. arXiv.
17. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020). End-to-end object detection with transformers. European Conference on Computer Vision, Springer International Publishing.10.1007/978-3-030-58452-8_13 ↗
18. Wang, L., Du, L., Ye, X., Fu, Y., Guo, G., Xue, X., Feng, J., and Zhang, L. (2021, January 20–25). Depth-conditioned dynamic message propagation for monocular 3d object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00052 ↗
19. Xu, B., and Chen, Z. (2018, January 18–23). Multi-level fusion based 3d object detection from monocular images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00249 ↗
20. Qi, C.R., Liu, W., Wu, C., Su, H., and Guibas, L.J. (2018, January 18–23). Frustum pointnets for 3d object detection from rgb-d data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00102 ↗
21. Reading, C., Harakeh, A., Chae, J., and Waslander, S.L. (2021, January 20–25). Categorical depth distribution network for monocular 3d object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00845 ↗
22. Zhang, R., Qiu, H., Wang, T., Guo, Z., Cui, Z., Qiao, Y., Li, H., and Gao, P. (2023, January 1–6). Monodetr: Depth-guided transformer for monocular 3d object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00840 ↗
23. MonoPSTR: Monocular 3D Object Detection with Dynamic Position & Scale-aware Transformer
24. Shen, F., Ye, H., Zhang, J., Wang, C., Han, X., and Yang, W. (2023). Advancing pose-guided image synthesis with progressive conditional diffusion models. arXiv.
25. Shen, F., Ye, H., Liu, S., Zhang, J., Wang, C., Han, X., and Yang, W. (2025, January 20–22). Boosting consistency in story visualization with rich-contextual conditional diffusion models. Proceedings of the AAAI Conference on Artificial Intelligence, Madrid, Spain.10.1609/aaai.v39i7.32728 ↗
26. Shen, F., Jiang, X., He, X., Ye, H., Wang, C., Du, X., Li, Z., and Tang, J. (2025, January 20–22). Imagdressing-v1: Customizable virtual dressing. Proceedings of the AAAI Conference on Artificial Intelligence, Madrid, Spain.10.1609/aaai.v39i7.32729 ↗
27. Imagpose: A unified conditional framework for pose-guided person generation
28. Zou, C., Wang, B., Hu, Y., Liu, J., Wu, Q., Zhao, Y., Li, B., Zhang, C., Zhang, C., and Wei, Y. (2021, January 20–25). End-to-end human object interaction detection with hoi transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01165 ↗
29. Attention is all you need
30. Ross, T.Y., and Dollár, G. (2017, January 21–26). Focal loss for dense object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.
31. Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
32. Wei, L., Zheng, C., and Hu, Y. (2023). Oriented object detection in aerial images based on the scaled smooth L1 loss function. Remote Sens., 15.10.3390/rs15051350 ↗
33. Liu, C., Yu, S., Yu, M., Wei, B., Li, B., Li, G., and Huang, W. (2021, January 5–8). Adaptive smooth L1 loss: A better way to regress scene texts with extreme aspect ratios. Proceedings of the 2021 IEEE Symposium on Computers and Communications (ISCC), IEEE, Athens, Greece.10.1109/ISCC53001.2021.9631466 ↗
34. 3d object proposals using stereo imagery for accurate object class detection10.1109/TPAMI.2017.2706685 ↗
35. Zou, Z., Ye, X., Du, L., Cheng, X., Tan, X., Zhang, L., Feng, J., Xue, X., and Ding, E. (2021, January 10–17). The devil is in the task: Exploiting reciprocal appearance-localization features for monocular 3d object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00271 ↗
36. Zhou, Y., He, Y., Zhu, H., Wang, C., Li, H., and Jiang, Q. (2021, January 20–25). Monocular 3d object detection: An extrinsic parameter free approach. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00747 ↗
37. Zhang, Y., Lu, J., and Zhou, J. (2021, January 20–25). Objects are different: Flexible monocular 3d object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00330 ↗
38. Lu, Y., Ma, X., Yang, L., Zhang, T., Liu, Y., Chu, Q., Yan, J., and Ouyang, W. (2021, January 10–17). Geometry uncertainty projection network for monocular 3d object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00310 ↗
39. Huang, K.-C., Wu, T.-H., Su, H.-T., and Hsu, W.H. (2022, January 18–24). Monodtr: Monocular 3d object detection with depth-aware transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00398 ↗
40. Lian, Q., Li, P., and Chen, X. (2022, January 18–24). Monojsg: Joint semantic and geometric cost volume for monocular 3d object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00114 ↗
41. Kumar, A., Brazil, G., Corona, E., Parchami, A., and Liu, X. (2022). Deviant: Depth equivariant network for monocular 3d object detection. European Conference on Computer Vision, Springer Nature.10.1007/978-3-031-20077-9_39 ↗
42. Li, Y., Chen, Y., He, J., and Zhang, Z. (2022). Densely constrained depth estimator for monocular 3d object detection. European Conference on Computer Vision, Springer Nature.10.1007/978-3-031-20077-9_42 ↗
43. Peng, L., Wu, X., Yang, Z., Liu, H., and Cai, D. (2022). Did-m3d: Decoupling instance depth for monocular 3d object detection. European Conference on Computer Vision, Springer Nature.10.1007/978-3-031-19769-7_5 ↗
44. Brazil, G., Kumar, A., Straub, J., Ravi, N., Johnson, J., and Gkioxari, G. (2023, January 17–24). Omni3d: A large benchmark and model for 3d object detection in the wild. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01264 ↗
45. Monouni: A unified vehicle and infrastructure-side monocular 3d object detection network with sufficient depth clues
46. Zhou, Y., Zhu, H., Liu, Q., Chang, S., and Guo, M. (2023, January 17–24). Monoatt: Online monocular 3d object detection with adaptive token transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01678 ↗
47. Zhang, J., Li, J., Lin, X., Zhang, W., Tan, X., Han, J., Ding, E., Wang, J., and Li, G. (2024). Decoupled Pseudo-Labeling for Semi-Supervised Monocular 3D Object Detection. arXiv.10.1109/CVPR52733.2024.01601 ↗
48. Dp-M3D: Monocular 3D object detection algorithm with depth perception capability10.1016/j.knosys.2025.113539 ↗
49. Fu, H., Gong, M., Wang, C., Batmanghelich, K., and Tao, D. (2018, January 18–23). Deep ordinal regression network for monocular depth estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00214 ↗