Dist-YOLO: Fast Object Detection with Distance Estimation
Source abstract
Abstract
We present a scheme of how YOLO can be improved in order to predict the absolute distance of objects using only information from a monocular camera. It is fully integrated into the original architecture by extending the prediction vectors, sharing the backbone’s weights with the bounding box regressor, and updating the original loss function by a part responsible for distance estimation. We designed two ways of handling the distance, class-agnostic and class-aware, proving class-agnostic creates smaller prediction vectors than class-aware and achieves better results. We demonstrate that the subtasks of object detection and distance measurement are in synergy, resulting in the increase of the precision of the original bounding box functionality. We show that using the KITTI dataset, the proposed scheme yields a mean relative error of 11% considering all eight classes and the distance range within [0, 150] m, which makes the solution highly competitive with existing approaches. Finally, we show that the inference speed is identical to the unmodified YOLO, 45 frames per second.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Advanced Vision and Imaging · Image Processing Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Centre of Excellence IT4Innovations, Institute for Research and Applications of Fuzzy Modeling, 30. Dubna 22, University of Ostrava, 702 00 Ostrava, Czech Republic
Varroc Lighting Systems, Suvorovova 195, 742 42 Šenov u Nového Jičína, Czech Republic
References (53)
Show source-provided references
1. Rukhovich, D., Mouritzen, D., Kaestner, R., Rufli, M., and Velizhev, A. (2019, January 27–28). Estimation of Absolute Scale in Monocular SLAM Using Synthetic Data. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea.10.1109/ICCVW.2019.00108 ↗
2. Haseeb, M.A., Guan, J., Ristic-Durrant, D., and Gräser, A. (2018, January 1). DisNet: A novel method for distance estimation from monocular camera. Proceedings of the 10th Planning, Perception and Navigation for Intelligent Vehicles (PPNIV18), Madrid, Spain.
3. Zhang, K., Xie, J., Snavely, N., and Chen, Q. (2020, January 13–19). Depth sensing beyond lidar range. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00176 ↗
4. Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.
5. Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv.
6. Jocher, G., Stoken, A., Borovec, J., NanoCode012, ChristopherSTAN, Liu, C., Laughing, Tkianai, Hogan, A., and Lorenzomammana (2022, January 26). Ultralytics/yolov5: v3.1-Bug Fixes and Performance Improvements. Available online: https://zenodo.org/record/4154370#.YfJaRfgRXQw.
7. Monocular distance estimation from optic flow during active landing maneuvers10.1088/1748-3182/9/2/025002 ↗
8. Ranftl, R., Bochkovskiy, A., and Koltun, V. (2021). Vision Transformers for Dense Prediction. arXiv.10.1109/ICCV48922.2021.01196 ↗
9. Consistent video depth estimation10.1145/3386569.3392377 ↗
10. Kumari, S., Jha, R.R., Bhavsar, A., and Nigam, A. (2019, January 22–25). Autodepth: Single image depth map estimation via residual cnn encoder-decoder and stacked hourglass. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8803006 ↗
11. As-planar-as-possible depth map estimation10.1016/j.cviu.2019.02.001 ↗
12. Zhou, Y., and Tuzel, O. (2018, January 18–23). Voxelnet: End-to-end learning for point cloud based 3d object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00472 ↗
13. Lehner, J., Mitterecker, A., Adler, T., Hofmarcher, M., Nessler, B., and Hochreiter, S. (2019). Patch Refinement–Localized 3D Object Detection. arXiv.
14. Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16–21). Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA.10.1109/CVPR.2012.6248074 ↗
15. Geiger, A., Lenz, P., and Urtasun, R. (2019, July 14). KITTI 3D Object Detection Benchmark. Available online: http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d.
16. Saxena, A., Chung, S.H., and Ng, A.Y. (2006). Learning depth from single monocular images. Advances in Neural Information Processing Systems, MIT Press.
17. Garg, R., BG, V.K., Carneiro, G., and Reid, I. (2016). Unsupervised cnn for single view depth estimation: Geometry to the rescue. European Conference on Computer Vision, Springer.10.1007/978-3-319-46484-8_45 ↗
18. Zhu, J., and Fang, Y. (November, January 27). Learning object-specific distance from a monocular image. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.
19. Hu, H.N., Cai, Q.Z., Wang, D., Lin, J., Sun, M., Krahenbuhl, P., Darrell, T., and Yu, F. (November, January 27). Joint monocular 3D vehicle detection and tracking. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.
20. Robust inter-vehicle distance estimation method based on monocular vision10.1109/ACCESS.2019.2907984 ↗
21. Natanael, G., Zet, C., and Foşalău, C. (2018, January 18–19). Estimating the distance to an object based on image processing. Proceedings of the 2018 International Conference and Exposition on Electrical Furthermore, Power Engineering (EPE), Iasi, Romania.10.1109/ICEPE.2018.8559642 ↗
22. Haseeb, M.A., Ristić-Durrant, D., and Gräser, A. (2018, January 13–14). Long-range obstacle detection from a monocular camera. Proceedings of the ACM Computer Science in Cars Symposium (CSCS), Munich, Germany.
23. Chen, Z., Khemmar, R., Decoux, B., Atahouet, A., and Ertaud, J.Y. (2019, January 22–24). Real time object detection, tracking, and distance and motion estimation based on deep learning: Application to smart mobility. Proceedings of the 2019 Eighth International Conference on Emerging Security Technologies (EST), Colchester, UK.10.1109/EST.2019.8806222 ↗
24. Godard, C., Mac Aodha, O., and Brostow, G.J. (2017, January 21–26). Unsupervised monocular depth estimation with left-right consistency. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.699 ↗
25. Strbac, B., Gostovic, M., Lukac, Z., and Samardzija, D. (2020, January 26–27). YOLO Multi-Camera Object Detection and Distance Estimation. Proceedings of the 2020 Zooming Innovation in Consumer Technologies Conference (ZINC), Novi Sad, Serbia.10.1109/ZINC50678.2020.9161805 ↗
26. Mauri, A., Khemmar, R., Decoux, B., Haddad, M., and Boutteau, R. (2021). Real-time 3D multi-object detection and localization based on deep learning for road and railway smart mobility. J. Imaging, 7.10.3390/jimaging7080145 ↗
27. You, Y., Wang, Y., Chao, W.L., Garg, D., Pleiss, G., Hariharan, B., Campbell, M., and Weinberger, K.Q. (2019). Pseudo-lidar++: Accurate depth for 3d object detection in autonomous driving. arXiv.
28. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
29. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
30. Ge, Z., Liu, S., Wang, F., Li, Z., and Sun, J. (2021). Yolox: Exceeding yolo series in 2021. arXiv.
31. Hurtik, P., Molek, V., Hula, J., Vajgl, M., Vlasanek, P., and Nejezchleba, T. (2020). Poly-YOLO: Higher speed, more precise detection and instance segmentation for YOLOv3. arXiv.
32. Bolya, D., Zhou, C., Xiao, F., and Lee, Y.J. (2019). YOLACT: Real-time Instance Segmentation. arXiv.10.1109/ICCV.2019.00925 ↗
33. YOLACT++: Better Real-time Instance Segmentation10.1109/TPAMI.2020.3014297 ↗
34. Faster r-cnn: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
35. Yu, J., and Zhang, W. (2021). Face mask wearing detection algorithm based on improved YOLO-v4. Sensors, 21.10.3390/s21093263 ↗
36. Roy, A.M., Bose, R., and Bhaduri, J. (2022). A fast accurate fine-grain object detection model based on YOLOv4 deep neural network. Neural Comput. Appl., 1–27.10.1007/s00521-021-06651-x ↗
37. An improved tiny-yolov3 pedestrian detection algorithm10.1016/j.ijleo.2019.02.038 ↗
38. Jiao, Z., Zhang, Y., Xin, J., Mu, L., Yi, Y., Liu, H., and Liu, D. (2019, January 23–27). A deep learning based forest fire detection approach using UAV and YOLOv3. Proceedings of the 2019 1st International Conference on Industrial Artificial Intelligence (IAI), Shenyang, China.10.1109/ICIAI.2019.8850815 ↗
39. Zhou, F., Zhao, H., and Nie, Z. (2021, January 22–24). Safety Helmet Detection Based on YOLOv5. Proceedings of the 2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA), Shenyang, China.10.1109/ICPECA51329.2021.9362711 ↗
40. De Guzman, S.R.C., Tan, L.C., and Villaverde, J.F. (August, January 30). Social Distancing Violation Monitoring Using YOLO for Human Detection. Proceedings of the 2021 IEEE 7th International Conference on Control Science and Systems Engineering (ICCSSE), Qingdao, China.10.1109/ICCSSE52761.2021.9545098 ↗
41. Rahim, A., Maqbool, A., and Rana, T. (2022). Monitoring social distancing under various low light conditions with deep learning and a single motionless time of flight camera. PLoS ONE, 16.10.1371/journal.pone.0247440 ↗
42. FMD-Yolo: An efficient face mask detection method for COVID-19 prevention and control in public10.1016/j.imavis.2021.104341 ↗
43. Ioffe, S., and Szegedy, C. (2015, January 7–9). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference of International Conference on Machine Learning, Lille, France.
44. Qi, C.R., Zhou, Y., Najibi, M., Sun, P., Vo, K., Deng, B., and Anguelov, D. (2021). Offboard 3D Object Detection from Point Cloud Sequences. arXiv.10.1109/CVPR46437.2021.00607 ↗
45. Yu, F., Xian, W., Chen, Y., Liu, F., Liao, M., Madhavan, V., and Darrell, T. (2018). Bdd100k: A diverse driving video database with scalable annotation tooling. arXiv.
46. Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., and Beijbom, O. (2019). nuScenes: A multimodal dataset for autonomous driving. arXiv.10.1109/CVPR42600.2020.01164 ↗
47. Lin, T.Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Dollár, P. (2015). Microsoft COCO: Common Objects in Context. arXiv.10.1007/978-3-319-10602-1_48 ↗
48. Lin, T.Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Dollár, P. (2021, October 15). Microsoft COCO—Common Objects in Context. Available online: https://cocodataset.org//#detection-eval.
49. Mordan, T., Thome, N., Henaff, G., and Cord, M. (2018, January 2–8). Revisiting multi-task learning with rock: A deep residual auxiliary block for visual detection. Proceedings of the 32nd Conference on Neural Information Processing Systems (NeurIPS), Montreal, QC, Canada.
50. Crowd detection in aerial images using spatial graphs and fully-convolutional neural networks10.1109/ACCESS.2020.2984768 ↗
51. Ali, A., Hassan, A., Ali, A.R., Khan, H.U., Kazmi, W., and Zaheer, A. (2020, January 1–5). Real-time vehicle distance estimation using single view geometry. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass, CO, USA.10.1109/WACV45572.2020.9093634 ↗
52. Fu, H., Gong, M., Wang, C., Batmanghelich, K., and Tao, D. (2018, January 18–23). Deep ordinal regression network for monocular depth estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00214 ↗
53. Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009, January 14–18). Curriculum learning. Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada.10.1145/1553374.1553380 ↗