Custom Anchorless Object Detection Model for 3D Synthetic Traffic Sign Board Dataset with Depth Estimation and Text Character Extraction
Source abstract
Abstract
This paper introduces an anchorless deep learning model designed for efficient analysis and processing of large-scale 3D synthetic traffic sign board datasets. With an ever-increasing emphasis on autonomous driving systems and their reliance on precise environmental perception, the ability to accurately interpret traffic sign information is crucial. Our model seamlessly integrates object detection, depth estimation, deformable parts, and text character extraction functionalities, facilitating a comprehensive understanding of road signs in simulated environments that mimic the real world. The dataset used has a large number of artificially generated traffic signs for 183 different classes. The signs include place names in Japanese and English, expressway names in Japanese and English, distances and motorway numbers, and direction arrow marks with different lighting, occlusion, viewing angles, camera distortion, day and night cycles, and bad weather like rain, snow, and fog. This was done so that the model could be tested thoroughly in a wide range of difficult conditions. We developed a convolutional neural network with a modified lightweight hourglass backbone using depthwise spatial and pointwise convolutions, along with spatial and channel attention modules that produce resilient feature maps. We conducted experiments to benchmark our model against the baseline model, showing improved accuracy and efficiency in both depth estimation and text extraction tasks, crucial for real-time applications in autonomous navigation systems. With its model efficiency and partwise decoded predictions, along with Optical Character Recognition (OCR), our approach suggests its potential as a valuable tool for developers of Advanced Driver-Assistance Systems (ADAS), Autonomous Vehicle (AV) technologies, and transportation safety applications, ensuring reliable navigation solutions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Vehicle License Plate Recognition · Infrastructure Maintenance and Monitoring
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Graduate School of Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, Japan
References (60)
Show source-provided references
1. BlenderProc2: A Procedural Pipeline for Photorealistic Rendering10.21105/joss.04901 ↗
2. Community, B.O. (2022, April 15). Blender—A 3D Modelling and Rendering Package. Stichting Blender Foundation, Amsterdam: Blender Foundation. Available online: https://www.blender.org.
3. Haas, J.K. (2014). A History of the Unity Game Engine. [Ph.D. Thesis, Worcester Polytechnic Institute]. Available online: https://www.unity.com.
4. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). Playing for Data: Ground Truth from Computer Games. Computer Vision—ECCV 2016, Proceedings of the 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016, Springer International Publishing. Lecture Notes in Computer Science.10.1007/978-3-319-46478-7 ↗
5. Tremblay, J., Prakash, A., Acuna, D., Brophy, M., Jampani, V., Anil, C., To, T., Cameracci, E., Boochoon, S., and Birchfield, S. (2018, January 18–22). Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.10.1109/CVPRW.2018.00143 ↗
6. Baek, Y., Lee, B., Han, D., Yun, S., and Lee, H. (2019, January 15–20). Character Region Awareness for Text Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00959 ↗
7. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft COCO: Common Objects in Context. Proceedings of the 13th European Conference on Computer Vision (ECCV), Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
8. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks10.1109/TPAMI.2016.2577031 ↗
9. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
10. CornerNet: Detecting Objects as Paired Keypoints10.1007/s11263-019-01204-1 ↗
11. Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., and Tian, Q. (November, January 27). CenterNet: Keypoint Triplets for Object Detection. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00667 ↗
12. Zhou, X., Zhuo, J., and Krahenbuhl, P. (2019, January 15–20). Bottom-up object detection by grouping extreme and center points. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00094 ↗
13. Zhou, X., Wang, D., and Krähenbühl, P. (2019). Objects as points. arXiv.
14. Machine Vision Based Traffic Sign Detection Methods: Review, Analyses and Perspectives10.1109/ACCESS.2019.2924947 ↗
15. Zakir, U., Leonce, A.N.J., and Edirisinghe, E. (2010, January 17–19). Road Sign Segmentation Based on Colour Spaces: A Comparative Study. Proceedings of the 11th Iasted International Conference on Computer Graphics and Imgaing, Innsbruck, Austria.10.2316/P.2010.679-061 ↗
16. Fast Traffic Sign Recognition Using Color Segmentation and Deep Convolutional Networks10.1007/978-3-319-48680-2_19 ↗
17. Prisacariu, V.A., Timofte, R., Zimmermann, K., Reid, I., and Van Gool, L. (2010, January 23–26). Integrating Object Detection with 3D Tracking towards a Better Driver Assistance System. Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey.10.1109/ICPR.2010.816 ↗
18. An Overview of Traffic Sign Detection and Classification Methods10.1007/s13735-017-0129-8 ↗
19. Rajendran, S.P., Shine, L., Pradeep, R., and Vijayaraghavan, S. (2019, January 6–8). Real-Time Traffic Sign Recognition Using YOLOv3 Based Detector. Proceedings of the 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kanpur, India.10.1109/ICCCNT45670.2019.8944890 ↗
20. Li, Y., Li, J., and Meng, P. (2022). Attention-YOLOV4: A Real-Time and High-Accurate Traffic Sign Detection Algorithm, Springer.10.1007/s11042-022-13251-x ↗
21. Newell, A., Yang, K., and Deng, J. (2016, January 11–14). Stacked Hourglass Networks for Human Pose Estimation. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46484-8_29 ↗
22. Soans, R.V., and Fukumizu, Y. (2022, January 21–22). Improved Facial Keypoint Regression Using Attention Modules. Proceedings of the Communi-cations in Computer and Information Science, Frontiers of Computer Vision, Hiroshima, Japan.10.1007/978-3-031-06381-7_13 ↗
23. Shivanna, V.M., and Guo, J. (2023). Object Detection, Recognition, and Tracking Algorithms for ADASs—A Study on Recent Trends. Sensors, 24.10.3390/s24010249 ↗
24. Object detection using YOLO: Challenges, architectural successors, datasets and applications10.1007/s11042-022-13644-y ↗
25. Improved YOLOv5 network for real-time multi-scale traffic sign detection10.1007/s00521-022-08077-5 ↗
26. Chu, J., Zhang, C., Yan, M., Zhang, H., and Ge, T. (2023). TRD-YOLO: A Real-Time, High-Performance Small Traffic Sign Detection Algorithm. Sensors, 23.10.3390/s23083871 ↗
27. Liu, H., Zhou, K., Zhang, Y., and Zhang, Y. (2023). ETSR-YOLO: An improved multi-scale traffic sign detection algorithm based on YOLOv5. PLoS ONE, 18.10.1371/journal.pone.0295807 ↗
28. Wang, Y., Bai, M., Wang, M., Zhao, F., and Guo, J. (2022). Multiscale Traffic Sign Detection Method in Complex Environment Based on YOLOv4. Computational Intelligence and Neuroscience, John Wiley & Sons Ltd.10.1155/2022/5297605 ↗
29. Liu, Y., Shi, G., Li, Y., and Zhao, Z. (2022). M-YOLO: Traffic Sign Detection Algorithm Applicable to Complex Scenarios. Symmetry, 14.10.3390/sym14050952 ↗
30. Shen, J., Zhang, Z., Luo, J., and Zhang, X. (2023). YOLOv5-TS: Detecting traffic signs in real-time. Front. Phys., 11.10.3389/fphy.2023.1297828 ↗
31. Zhang, K., Chen, J., Zhang, R., and Hu, C. (2024, January 22–24). A Hybrid Approach for Efficient Traffic Sign Detection Using Yolov8 and SAM. Proceedings of the 2024 3rd Asia Conference on Algorithms, Computing and Machine Learning, Shanghai, China.10.1145/3654823.3654878 ↗
32. A novel lightweight real-time traffic sign detection method based on an embedded device and YOLOv810.1007/s11554-023-01403-7 ↗
33. Traffic-light sign recognition using capsule network10.1007/s11042-020-10455-x ↗
34. Kumar, A.D. (2018). Novel Deep Learning Model for Traffic Sign Detection Using Capsule Networks. arXiv.
35. Optimizing Traffic Sign Detection and Recognition by Using Deep Learning10.18280/ijtdi.080112 ↗
36. Abraham, A., Pllana, S., Hanne, T., and Siarry, P. (2024). Intelligent Traffic Sign Detection and Recognition Using Computer Vision. Intelligent Systems Design and Applications, Springer.10.1007/978-3-031-64650-8 ↗
37. Detection and Identification of Text-based Traffic Signs
38. Kiefer, B., Ott, D., and Zell, A. (2022, January 21–25). Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles. Proceedings of the 2022 26th International Conference on Pattern Recognition (ICPR), Montreal, QC, Canada.10.1109/ICPR56361.2022.9956710 ↗
39. Premakumara, N., Jalaeian, B., Suri, N., and Samani, H.A. (2023). Enhancing object detection robustness: A synthetic and natural perturbation approach. arXiv.
40. Synthetic Data and Hierarchical Object Detection in Overhead Imagery10.1145/3635309 ↗
41. Adobe Inc. (2022, January 07). Adobe Illustrator. Available online: https://adobe.com/products/illustrator.
42. Adobe Inc. (2023, April 20). Adobe Photoshop. Available online: https://www.adobe.com/products/photoshop.html.
43. The GIMP Development Team (2022, January 07). GIMP. Available online: https://www.gimp.org.
44. Inkscape Project (2022, January 07). Inkscape. Available online: https://inkscape.org.
45. Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. (August, January 31). The German traffic sign recognition benchmark: A multi-class classification competition. Proceedings of the 2011 International Joint Conference on Neural Networks, San Jose, CA, USA.10.1109/IJCNN.2011.6033395 ↗
46. Towards real-time traffic sign detection and classification10.1109/TITS.2015.2482461 ↗
47. Wang, Q., Wu, B., Zhu, P.F., Li, P., Zuo, W., and Hu, Q. (2020, January 14–19). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01155 ↗
48. Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27–30). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.350 ↗
49. OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields10.1109/TPAMI.2019.2929257 ↗
50. Noh, H., Hong, S., and Han, B. (2015, January 7–13). Learning deconvolution network for semantic segmentation. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.178 ↗
51. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation10.1109/TPAMI.2016.2644615 ↗
52. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.
53. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
54. Hu, J., Shen, L., Albanie, S., Sun, G., and Wu, E. (2018, January 18–22). Squeeze-and-Excitation Networks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
55. Graves, A., Fern’andez, S., Gomez, F.J., and Schmidhuber, J. (2006, January 25–29). Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA.10.1145/1143844.1143891 ↗
56. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
57. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition10.1109/TPAMI.2016.2646371 ↗
58. Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv.
59. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
60. Wang, C., Yeh, I., and Liao, H. (2024). YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information. arXiv.