A Survey on Theories and Applications for Self-Driving Cars Based on Deep Learning Methods
Source abstract
Abstract
Self-driving cars are a hot research topic in science and technology, which has a great influence on social and economic development. Deep learning is one of the current key areas in the field of artificial intelligence research. It has been widely applied in image processing, natural language understanding, and so on. In recent years, more and more deep learning-based solutions have been presented in the field of self-driving cars and have achieved outstanding results. This paper presents a review of recent research on theories and applications of deep learning for self-driving cars. This survey provides a detailed explanation of the developments of self-driving cars and summarizes the applications of deep learning methods in the field of self-driving cars. Then the main problems in self-driving cars and their solutions based on deep learning methods are analyzed, such as obstacle detection, scene recognition, lane detection, navigation and path planning. In addition, the details of some representative approaches for self-driving cars using deep learning methods are summarized. Finally, the future challenges in the applications of deep learning for self-driving cars are given out.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Autonomous Vehicle Technology and Safety · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of IOT Engineering, Hohai University, Changzhou 213022, China
Jiangsu Universities and Colleges Key Laboratory of Special Robot Technology, Hohai University, Changzhou 213022, China
Funding
National Natural Science Foundation of China
References (94)
Show source-provided references
1. Duarte, F. (2019). Self-driving cars: A city perspective. Sci. Robot., 4.10.1126/scirobotics.aav9843 ↗
2. Unmanned aerial vehicles (UAVs): A survey on civil applications and key research challenges10.1109/ACCESS.2019.2909530 ↗
3. Economic effects of automated vehicles10.3141/2606-14 ↗
4. Autonomous cars: research results, issues, and future challenges10.1109/COMST.2018.2869360 ↗
5. A survey of scene understanding by event reasoning in autonomous driving10.1007/s11633-018-1126-y ↗
6. Wang, Z., Liu, K., Li, J., Zhu, Y., and Zhang, Y. (2019). Various frameworks and libraries of machine learning and deep learning: A survey. Arch. Comput. Methods Eng.10.1007/s11831-018-09312-w ↗
7. A survey on deep learning based face recognition10.1016/j.cviu.2019.102805 ↗
8. Self-driving cars and the law10.1109/MSPEC.2016.7419800 ↗
9. Bengio, Y.I., Goodfellow, J., and Courville, A. (2016). Deep Learning, MIT Press.
10. Google and ITE: The road ahead for self-driving cars
11. Dikmen, M., and Burns, C. (2017, January 5–8). Trust in autonomous vehicles: The case of Tesla autopilot and summon. Proceedings of the 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017, Banff, AB, Canada.10.1109/SMC.2017.8122757 ↗
12. Driving tests for self-driving cars10.1109/MSPEC.2018.8302386 ↗
13. Exploring the key services and players in the smart car market10.1016/j.telpol.2019.04.003 ↗
14. Toschi, A., Sanic, M., Leng, J., Chen, Q., Wang, C., and Guo, M. (2019, January 23–24). Characterizing perception module performance and robustness in production-scale autonomous driving system. Proceedings of the 16th IFIP WG 10.3 International Conference on Network and Parallel Computing, Hohhot, China.10.1007/978-3-030-30709-7_19 ↗
15. Li, Y., Wang, J., Xing, T., Liu, T., Li, C., and Su, K. (2017, January 17–20). TAD16K: An enhanced benchmark for autonomous driving. Proceedings of the 24th IEEE International Conference on Image Processing, Beijing, China.10.1109/ICIP.2017.8296701 ↗
16. Assessment of the safety benefits of vehicles advanced driver assistance, connectivity and low level automation systems10.1016/j.aap.2018.04.002 ↗
17. Fuzzy-based driver monitoring system (FDMS): Implementation of two intelligent FDMSs and a testbed for safe driving in VANETs10.1016/j.future.2019.12.030 ↗
18. A feedforward and feedback integrated lateral and longitudinal driver model for personalized advanced driver assistance systems10.1016/j.mechatronics.2018.02.007 ↗
19. A survey of motion planning and control techniques for self-driving urban vehicles10.1109/TIV.2016.2578706 ↗
20. A Survey on optimized implementation of deep learning models on the NVIDIA Jetson platform10.1016/j.sysarc.2019.01.011 ↗
21. Driverless when and cars: Where?10.1049/et.2017.0201 ↗
22. Deep learning for big data applications in CAD and PLM research review, opportunities and case study10.1016/j.compind.2018.04.005 ↗
23. A memetic algorithm with variable length chromosome for robot path planning under dynamic environments
24. Bioinspired intelligent algorithm and its applications for mobile robot control: A survey10.1155/2016/3810903 ↗
25. A taxonomy of autonomous vehicle handover situations10.1016/j.tra.2018.05.005 ↗
26. Accelerating geostatistical seismic inversion using TensorFlow: A heterogeneous distributed deep learning framework10.1016/j.cageo.2018.12.007 ↗
27. A survey on deep learning-based fine-grained object classification and semantic segmentation10.1007/s11633-017-1053-3 ↗
28. Spectral-spatial residual network for hyperspectral image classification: A 3-D deep learning framework10.1109/TGRS.2017.2755542 ↗
29. Modulation filter learning using deep variational networks for robust speech recognition10.1109/JSTSP.2019.2913965 ↗
30. Deep learning for environmentally robust speech recognition: An overview of recent developments10.1145/3178115 ↗
31. Recent trends in deep learning based natural language processing10.1109/MCI.2018.2840738 ↗
32. A review of natural language processing techniques for opinion mining systems10.1016/j.inffus.2016.10.004 ↗
33. A review of recurrent neural networks: LSTM cells and network architectures10.1162/neco_a_01199 ↗
34. Gers, F.A., Schmidhuber, J., and Cummins, F. (1999, January 7–10). Learning to forget: Continual prediction with LSTM. Proceedings of the 1999 the 9th International Conference on ‘Artificial Neural Networks (ICANN99)’, Edinburgh, UK.10.1049/cp:19991218 ↗
35. Deep neural network concepts for background subtraction: A systematic review and comparative evaluation10.1016/j.neunet.2019.04.024 ↗
36. Contextual deconvolution network for semantic segmentation10.1016/j.patcog.2019.107152 ↗
37. Woo, J., and Kim, N. (2020). Collision avoidance for an unmanned surface vehicle using deep reinforcement learning. Ocean Eng., in press.10.1016/j.oceaneng.2020.107001 ↗
38. Intelligent fault diagnosis for rotating machinery using deep Q-network based health state classification: A deep reinforcement learning approach10.1016/j.aei.2019.100977 ↗
39. A multiagent Q-learning-based optimal allocation approach for urban water resource management system10.1109/TASE.2012.2229978 ↗
40. Bioinspired neural network-based Q-learning approach for robot path planning in unknown environments
41. Wen, S., Zhao, Y., Yuan, X., Wang, Z., Zhang, D., and Manfredi, L. (2020). Path planning for active SLAM based on deep reinforcement learning under unknown environments. Intell. Serv. Robot., in press.10.1007/s11370-019-00310-w ↗
42. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
43. Robotic material perception using active multimodal fusion10.1109/TIE.2018.2878157 ↗
44. Radar and vision fusion for the real-time obstacle detection and identification10.1108/IR-06-2018-0113 ↗
45. Quantitative evaluation of stereo visual odometry for autonomous vessel localisation in inland waterway sensing applications10.3390/s151229892 ↗
46. J-MOD2: Joint monocular obstacle detection and depth estimation10.1109/LRA.2018.2800083 ↗
47. Monocular vision-based obstacle detection and avoidance for a multicopter10.1109/ACCESS.2019.2953954 ↗
48. DeepRange: Deep-learning-based object detection and ranging in autonomous driving10.1049/iet-its.2018.5144 ↗
49. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms10.1023/A:1014573219977 ↗
50. Stereo matching by training a convolutional neural network to compare image patches
51. Wide context learning network for stereo matching10.1016/j.image.2019.07.008 ↗
52. Multi-dimensional residual dense attention network for stereo matching10.1109/ACCESS.2019.2911618 ↗
53. Kendall, A., Martirosyan, H., Dasgupta, S., Henry, P., Kennedy, R., Bachrach, A., and Bry, A. (2017, January 22–29). End-to-end learning of geometry and context for deep stereo regression. Proceedings of the 16th IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy.10.1109/ICCV.2017.17 ↗
54. Obstacle detection for intelligent transportation systems using deep stacked autoencoder and k-nearest neighbor scheme10.1109/JSEN.2018.2831082 ↗
55. Mancini, M., Costante, G., Valigi, P., and Ciarfuglia, T.A. (2016, January 9–14). Fast robust monocular depth estimation for obstacle detection with fully convolutional networks. Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2016, Daejeon, Korea.10.1109/IROS.2016.7759632 ↗
56. Obstacle detection in single images with deep neural networks10.1007/s11760-015-0855-4 ↗
57. Zhong, Y., Li, H., and Dai, Y. (2018, January 8–14). Open-world stereo video matching with deep RNN. Proceedings of the 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany.10.1007/978-3-030-01216-8_7 ↗
58. Jie, Z., Wang, P., Ling, Y., Zhao, B., Wei, Y., Feng, J., and Liu, W. (2018, January 18–22). Left-right comparative recurrent model for stereo matching. Proceedings of the 31st Meeting of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00404 ↗
59. Deep integration: A multi-label architecture for road scene recognition10.1109/TIP.2019.2913079 ↗
60. Knowledge guided disambiguation for large-scale scene classification with multi-resolution CNNs10.1109/TIP.2017.2675339 ↗
61. G-MS2F: GoogLeNet based multi-stage feature fusion of deep CNN for scene recognition10.1016/j.neucom.2016.11.023 ↗
62. Byeon, W., Breuel, T.M., Raue, F., and Liwicki, M. (2015, January 7–12). Scene labeling with LSTM recurrent neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA.10.1109/CVPR.2015.7298977 ↗
63. Multi-level ensemble network for scene recognition10.1007/s11042-019-07933-2 ↗
64. A novel scene classification model combining ResNet based transfer learning and data augmentation with a filter10.1016/j.neucom.2019.01.090 ↗
65. Scene recognition with objectness10.1016/j.patcog.2017.09.025 ↗
66. Fusing object semantics and deep appearance features for scene recognition10.1109/TCSVT.2018.2848543 ↗
67. Real-time road surface and semantic lane estimation using deep features10.1007/s11760-018-1264-2 ↗
68. Xiao, D., Yang, X., Li, J., and Islam, M. (2020). Attention deep neural network for lane marking detection. Knowl. Based Syst., 105584.10.1016/j.knosys.2020.105584 ↗
69. Fast learning method for convolutional neural networks using extreme learning machine and its application to lane detection10.1016/j.neunet.2016.12.002 ↗
70. Learning full-reference quality-guided discriminative gradient cues for lane detection based on neural networks10.1016/j.jvcir.2019.102675 ↗
71. Deep neural network for structural prediction and lane detection in traffic scene10.1109/TNNLS.2016.2522428 ↗
72. Lane marking detection via deep convolutional neural network10.1016/j.neucom.2017.09.098 ↗
73. Neven, D., De Brabandere, B., Georgoulis, S., Proesmans, M., and Van Gool, L. (2018, January 26–30). Towards end-to-end lane detection: An instance segmentation approach. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium, IV 2018, Changshu, China.10.1109/IVS.2018.8500547 ↗
74. Improved lane detection with multilevel features in branch convolutional neural networks10.1109/ACCESS.2019.2957053 ↗
75. Robust lane detection from continuous driving scenes using deep neural networks10.1109/TVT.2019.2949603 ↗
76. Ghafoorian, M., Nugteren, C., Baka, N., Booij, O., and Hofmann, M. (2018, January 8–14). EL-GAN: Embedding loss driven generative adversarial networks for lane detection. Proceedings of the 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany.
77. A dynamic bioinspired neural network based real-time path planning method for autonomous underwater vehicles10.1155/2017/9269742 ↗
78. An improved spinal neural system-based approach for heterogeneous AUVs cooperative hunting10.1007/s40815-017-0395-x ↗
79. Intelligent land-vehicle model transfer trajectory planning method based on deep reinforcement learning10.3390/s18092905 ↗
80. Eraqi, H.M., Moustafa, M.N., and Honer, J. (2017, January 4–9). End-to-end deep learning for steering autonomous vehicles considering temporal dependencies. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), MLITS Workshop, Long Beach, CA, USA.
81. Multi-receptive field graph convolutional neural networks for pedestrian detection10.1049/iet-its.2018.5618 ↗
82. Xu, H., and Srivastava, G. (2020). Automatic recognition algorithm of traffic signs based on convolution neural network. Multimed. Tools Appl., in press.10.1007/s11042-019-08239-z ↗
83. Autonomous traffic sign (ATSR) detection and recognition using deep CNN10.1016/j.procs.2019.12.108 ↗
84. Accurate traffic light detection using deep neural network with focal regression loss10.1016/j.imavis.2019.04.003 ↗
85. Kim, H.K., Yoo, K.Y., Park, J.H., and Jung, H.Y. (2019). Traffic light recognition based on binary semantic segmentation network. Sensors, 19.10.3390/s19071700 ↗
86. Letter perception emerges from unsupervised deep learning and recycling of natural image features10.1038/s41562-017-0186-2 ↗
87. Fraga-Lamas, P., Ramos, L., Mondejar-Guerra, V., and Fernandez-Carames, T.M. (2019). A review on IoT deep learning UAV systems for autonomous obstacle detection and collision avoidance. Remote Sens., 11.10.3390/rs11182144 ↗
88. Micucci, A., Mantecchini, L., and Sangermano, M. (2019). Analysis of the relationship between turning signal detection and motorcycle driver’s characteristics on urban roads; A case study. Sensors, 19.10.20944/preprints201903.0070.v1 ↗
89. A study on cyclists behaviour and bicycles kinematic10.2495/TDI-V4-N1-14-28 ↗
90. A systematic review on deep learning architectures and applications10.1111/exsy.12400 ↗
91. A review of deep learning methods and applications for unmanned aerial vehicles10.1155/2017/3296874 ↗
92. Fusion of images and point clouds for the semantic segmentation of large-scale 3D scenes based on deep learning10.1016/j.isprsjprs.2018.04.022 ↗
93. Yang, Y., Chen, F., Wu, F., Zeng, D., Ji, Y.M., and Jing, X.Y. (2020). Multi-view semantic learning network for point cloud based 3D object detection. Neurocomputing, in press.10.1016/j.neucom.2019.10.116 ↗
94. Convolutional neural networks for 5G-enabled intelligent transportation system: A systematic review10.1016/j.comcom.2020.01.058 ↗