Autonomous Navigation of Robots: Optimization with DQN
Source abstract
Abstract
In the field of artificial intelligence, control systems for mobile robots have undergone significant advancements, particularly within the realm of autonomous learning. However, previous studies have primarily focused on predefined paths, neglecting real-time obstacle avoidance and trajectory reconfiguration. This research introduces a novel algorithm that integrates reinforcement learning with the Deep Q-Network (DQN) to empower an agent with the ability to execute actions, gather information from a simulated environment in Gazebo, and maximize rewards. Through a series of carefully designed experiments, the algorithm’s parameters were meticulously configured, and its performance was rigorously validated. Unlike conventional navigation systems, our approach embraces the exploration of the environment, facilitating effective trajectory planning based on acquired knowledge. By leveraging randomized training conditions within a simulated environment, the DQN network exhibits superior capabilities in computing complex functions compared to traditional methods. This breakthrough underscores the potential of our algorithm to significantly enhance the autonomous learning capacities of mobile robots.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotic Path Planning Algorithms · Reinforcement Learning in Robotics · Modular Robots and Swarm Intelligence
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Systems, Electronics and Industrial Engineering, Universidad Tecnica de Ambato (UTA), Ambato 180206, Ecuador
Electronics and Automation Department, Universidad Politecnica Salesiana (UPS), Quito 170146, Ecuador
Department of Systems Engineering, Automation and Industrial Informatics, Universitat Politecnica de Catalunya (UPC), 08034 Barcelona, Spain
Funding
Universidad Técnica de Ambato
References (100+)
Show source-provided references
1. Oxygen-vacancy-rich molybdenum carbide MXene nanonetworks for ultrasound-triggered and capturing-enhanced sonocatalytic bacteria eradication10.1016/j.biomaterials.2023.122074 ↗
2. The key factors of the industrial revolution 4.0 in the Malaysian smart manufacturing context10.4018/IJABIM.20220701.oa6 ↗
3. Zou, T., Situ, W., Yang, W., Zeng, W., and Wang, Y. (2023). A Method for Long-Term Target Anti-Interference Tracking Combining Deep Learning and CKF for LARS Tracking and Capturing. Remote Sens., 15.10.3390/rs15030748 ↗
4. Impact of Industry 4.0: Improving Hybrid Laser-Arc Welding with Big Data for Subsequent Functionality in UnderwaterWelding10.1007/978-3-030-68663-5_6 ↗
5. Dynamic Path Planning Based on Neural Networks for Aerial Inspection10.1007/s40313-022-00941-y ↗
6. Mizoguchi, Y., Hamada, D., Fukuda, R., Inniyaka, I., Kuwata, K., Nishimuta, K., Sugino, A., Tanaka, R., Yoshiki, T., and Nishida, Y. (2023). Image-based navigation of Small-size Autonomous Underwater Vehicle “Kyubic” in International Underwater Robot Competition, ALife Robotics Corporation Ltd.10.5954/ICAROB.2023.OS20-2 ↗
7. Control of Open Mobile Robotic Platform Using Deep Reinforcement Learning10.1007/978-3-031-24291-5_29 ↗
8. Enhancing manufacturing intelligence through an unsupervised data-driven methodology for cyclic industrial processes10.1016/j.eswa.2021.115269 ↗
9. Deep Learning for Victims Detection from Virtual and Real Search and Rescue Environments10.1007/978-3-031-21062-4_1 ↗
10. Object Segmentation for Bin Picking Using Deep Learning10.1007/978-3-031-21062-4_5 ↗
11. Intelligent Wheelchairs Rolling in Pairs Using Reinforcement Learning10.1007/978-3-031-21062-4_23 ↗
12. A machine learning framework with dataset-knowledgeability pre-assessment and a local decision-boundary crispness score: An industry 4.0-based case study on composite autoclave manufacturing10.1016/j.compind.2021.103510 ↗
13. An Approach to 3D Object Detection in Real-Time for Cognitive Robotics Experiments10.1007/978-3-031-21065-5_24 ↗
14. An autonomous learning mobile robot using biological reward modulate STDP10.1016/j.neucom.2021.06.027 ↗
15. Artificial Potential Field Incorporated Deep-Q-Network Algorithm for Mobile Robot Path Prediction10.32604/iasc.2023.028126 ↗
16. Raja, V., Talwar, D., Manchikanti, A., and Jha, S. (2023). Industry 4.0 and Advanced Manufacturing: Proceedings of I-4AM 2022, Springer. Lecture Notes in Mechanical Engineering.
17. Robotic Fertilization in Strip Cropping using a CNN Vegetables Detection-Characterization Method10.1016/j.compag.2022.106684 ↗
18. Herr, G., Weerakoon, L., Yu, M., and Chopra, N. (2022). Cardynet: Deep Learning Based Navigation for Car-Like Robots in Dynamic Environments, American Society of Mechanical Engineers (ASME).10.1115/IMECE2022-96023 ↗
19. Jaiswal, A., Ashutosh, K., Rousseau, J., Peng, Y., Wang, Z., and Ding, Y. (2022). RoS-KD: A Robust Stochastic Knowledge Distillation Approach for Noisy Medical Imaging, Institute of Electrical and Electronics Engineers Inc.10.1109/ICDM54844.2022.00118 ↗
20. Chen, C.W., Tsai, A.C., Zhang, Y.H., and Wang, J.F. (2022). 3D Object Detection Combined with Inverse Kinematics to Achieve Robotic Arm Grasping, Institute of Electrical and Electronics Engineers Inc.10.1109/ICOT56925.2022.10008135 ↗
21. Kulkarni, J., and Pantawane, P. (2022). Person Following Robot Based on Real Time Single Object Tracking and RGB-D Image, Institute of Electrical and Electronics Engineers Inc.10.1109/ICoNSIP49665.2022.10007460 ↗
22. M’Sila, C., Ayad, R., and Ait-Oufroukh, N. (2022). Automated Foreign Object Debris Detection System Based on UAV, Institute of Electrical and Electronics Engineers Inc.10.1109/ICNSC55942.2022.10004050 ↗
23. Balachandran, A., Lal S, A., and Sreedharan, P. (2022). Autonomous Navigation of an AMR Using Deep Reinforcement Learning in a Warehouse Environment, Institute of Electrical and Electronics Engineers Inc.10.1109/MysuruCon55714.2022.9971804 ↗
24. Ghodake, A., Uttam, P., and Ahuja, B. (2022). Accurate 6-DOF Grasp Pose Detection in Cluttered Environments Using Deep Learning, Institute of Electrical and Electronics Engineers Inc.10.1109/I4Tech55392.2022.9952955 ↗
25. Zhang, J., Xu, Z., Wu, J., Chen, Q., and Wang, F. (2022). Lightweight Intelligent Autonomous Unmanned Vehicle Based on Deep Neural Network in ROS System, Institute of Electrical and Electronics Engineers Inc.10.1109/ICISCAE55891.2022.9927506 ↗
26. Miyama, M. (2022, January 28–30). Robust inference of multi-task convolutional neural network for advanced driving assistance by embedding coordinates. Proceedings of the 8th World Congress on Electrical Engineering and Computer Systems and Science, EECSS 2022, Prague, Czech Republic.10.11159/mvml22.105 ↗
27. Jebbar, M., Maizate, A., and Ait Abdelouahid, R. (2022). Moroccan’s Arabic Speech Training And Deploying Machine Learning Models with Teachable Machine, Elsevier.10.1016/j.procs.2022.07.120 ↗
28. Copot, C., Shi, L., Smet, E., Ionescu, C., and Vanlanduit, S. (2022). Comparison of Deep Learning Models in Position Based Visual Servoing, Institute of Electrical and Electronics Engineers Inc.10.1109/ETFA52439.2022.9921442 ↗
29. Liu, J., Rangwala, M., Ahluwalia, K., Ghajar, S., Dhami, H., Tokekar, P., Tracy, B., and Williams, R. (2022). Intermittent Deployment for Large-Scale Multi-Robot Forage Perception: Data Synthesis, Prediction, and Planning. IEEE Trans. Autom. Sci. Eng., 1–21.10.1109/TASE.2022.3211873 ↗
30. Real-Time Detection of Ripe Oil Palm Fresh Fruit Bunch Based on YOLOv410.1109/ACCESS.2022.3204762 ↗
31. Lin, H.Z., Chen, H.H., Choophutthakan, K., and Li, C.H. (2022). Autonomous Mobile Robot as a Cyber-Physical System Featuring Networked Deep Learning and Control, Institute of Electrical and Electronics Engineers Inc.10.1109/AIM52237.2022.9863365 ↗
32. Mandel, N., Sandino, J., Galvez-Serna, J., Vanegas, F., Milford, M., and Gonzalez, F. (2022). Resolution-adaptive Quadtrees for Semantic Segmentation Mapping in UAV Applications, IEEE Computer Society.10.1109/AERO53065.2022.9843498 ↗
33. The Method for Automatic Adjustment of AGV’s PID Based on Deep Reinforcement Learning
34. The Determination of Reward Function in AGV Motion Control Based on DQN
35. Chavez-Galaviz, J., and Mahmoudian, N. (2022). Underwater Dock Detection through Convolutional Neural Networks Trained with Artificial Image Generation, Institute of Electrical and Electronics Engineers Inc.10.1109/ICRA46639.2022.9812143 ↗
36. Carvalho, E., Susbielle, P., Hably, A., Dibangoye, J., and Marchand, N. (2022). Neural Enhanced Control for Quadrotor Linear Behavior Fitting, Institute of Electrical and Electronics Engineers Inc.10.1109/ICUAS54217.2022.9836058 ↗
37. Liu, K., Zhou, X., Zhao, B., Ou, H., and Chen, B. (2022, January 27–30). An Integrated Visual System for Unmanned Aerial Vehicles Following Ground Vehicles: Simulations and Experiments. Proceedings of the 2022 IEEE 17th International Conference on Control & Automation (ICCA), 2022 IEEE 17th International Conference on Control & Automation (ICCA), Naples, Italy.10.1109/ICCA54724.2022.9831831 ↗
38. Yun, J., Jiang, D., Sun, Y., Huang, L., Tao, B., Jiang, G., Kong, J., Weng, Y., Li, G., and Fang, Z. (2022). Grasping Pose Detection for Loose Stacked Object Based on Convolutional Neural Network with Multiple Self-Powered Sensors Information. IEEE Sens. J.10.1109/JSEN.2022.3190560 ↗
39. Zhu, C., Chen, L., Cai, Y., Wang, H., and Li, Y. (2022, January 25–27). Vehicle-Mounted Multi-Object Tracking Based on Self-Query. Proceedings of the International Conference on Advanced Algorithms and Neural Networks (AANN 2022), Zhuhai, China.
40. Segmentation of Drug-Treated Cell Image and Mitochondrial-Oxidative Stress Using Deep Convolutional Neural Network10.1155/2022/5641727 ↗
41. Saripuddin, M., Suliman, A., and Sameon, S. (2022). Impact of Resampling and Deep Learning to Detect Anomaly in Imbalance Time-Series Data, Institute of Electrical and Electronics Engineers Inc.10.1109/ICCRD54409.2022.9730424 ↗
42. Intelligent transformation of the manufacturing industry for Industry 4.0: Seizing financial benefits from supply chain relationship capital through enterprise green management10.1016/j.techfore.2021.120999 ↗
43. A Data-Driven Cyber-Physical System Using Deep-Learning Convolutional Neural Networks: Study on False-Data Injection Attacks in an Unmanned Ground Vehicle Under Fault-Tolerant Conditions10.1109/TSMC.2022.3170071 ↗
44. Sinulingga, H.R., and Munir, R. (2020, January 8–9). Road Recognition System with Heuristic Method and Machine Learning. Proceedings of the 2020 7th International Conference on Advance Informatics: Concepts, Theory and Applications (ICAICTA), Tokoname, Japan.10.1109/ICAICTA49861.2020.9429073 ↗
45. Bhattacharya, S., Dutta, S., Maiti, T.K., Miura-Mattausch, M., Navarro, D., and Mattausch, H.J. (2018, January 29–31). Machine learning algorithm for autonomous control of walking robot. Proceedings of the 2018 International Symposium on Devices, Circuits and Systems (ISDCS), Howrah, India.10.1109/ISDCS.2018.8379644 ↗
46. Footstep planning of humanoid robot in ROS environment using Generative Adversarial Networks (GANs) deep learning10.1016/j.robot.2022.104269 ↗
47. Collision Avoidance Route Planning for Autonomous Medical Devices Using Multiple Depth Cameras10.1109/ACCESS.2022.3159239 ↗
48. Ranganathan, G., Bestak, R., Palanisamy, R., and Rocha, Á. (2022). Pervasive Computing and Social Networking, Springer. Lecture Notes in Networks and Systems.10.1007/978-981-16-5640-8 ↗
49. Optimization and improvement of a robotics gaze control system using LSTM networks10.1007/s11042-021-11112-7 ↗
50. Caesarendra, W., Wijaya, T., Pappachan, B.K., and Tjahjowidodo, T. (2019, January 18). Adaptation to industry 4.0 using machine learning and cloud computing to improve the conventional method of deburring in aerospace manufacturing industry. Proceedings of the 2019 International Conference on Information and Communication Technology and Systems, ICTS 2019, Surabaya, Indonesia.10.1109/ICTS.2019.8850990 ↗
51. Supervisory Control-Based Navigation Architecture: A New Framework for Autonomous Robots in Industry 4.0 Environments10.1109/TII.2017.2788079 ↗
52. Direct identification of ALK and ROS1 fusions in non-small cell lung cancer from hematoxylin and eosin-stained slides using deep learning algorithms10.1038/s41379-022-01141-4 ↗
53. Ma, C.Y., Zhou, J.Y., Xu, X.T., Qin, S.B., Han, M.F., Cao, X.H., Gao, Y.Z., Xu, L., Zhou, J.J., and Zhang, W. (2022). Clinical evaluation of deep learning–based clinical target volume three-channel auto-segmentation algorithm for adaptive radiotherapy in cervical cancer. BMC Med. Imaging, 22.10.1186/s12880-022-00851-0 ↗
54. Xin, J., Zhao, H., Liu, D., and Li, M. (2017, January 20–22). Application of deep reinforcement learning in mobile robot path planning. Proceedings of the Proceedings—2017 Chinese Automation Congress, CAC 2017, Jinan, China.10.1109/CAC.2017.8244061 ↗
55. Gattu, S., and Penumacha, K. (2022, January 14–16). Autonomous Navigation and Obstacle Avoidance using Self-Guided and Self-Regularized Actor-Critic. Proceedings of the 8th International Conference on Robotics and Artificial Intelligence, Singapore.10.1145/3573910.3573914 ↗
56. Single-model multi-tasks deep learning network for recognition and quantitation of surface-enhanced Raman spectroscopy10.1364/OE.472726 ↗
57. Bravo-Arrabal, J., Toscano-Moreno, M., Fernandez-Lozano, J., Mandow, A., Gomez-Ruiz, J., and Garcà a-Cerezo, A. (2021). The internet of cooperative agents architecture (X-ioca) for robots, hybrid sensor networks, and mec centers in complex environments: A search and rescue case study. Sensors, 21.10.3390/s21237843 ↗
58. Machine learning on properties of multiscale multisource hydroxyapatite nanoparticles datasets with different morphologies and sizes10.1038/s41524-021-00618-1 ↗
59. Effective data-balancing methods for class-imbalanced genotoxicity datasets using machine learning algorithms and molecular fingerprints10.1016/j.comtox.2021.100178 ↗
60. Development of Smart Mobile Manipulator Controlled by a Single Windows PC Equipped with Real-Time Control Software10.1007/s12541-021-00571-5 ↗
61. Hierarchical deep reinforcement learning to drag heavy objects by adult-sized humanoid robot10.1016/j.asoc.2021.107601 ↗
62. Chen, K., Liang, Y., Jha, N., Ichnowski, J., Danielczuk, M., Gonzalez, J., Kubiatowicz, J., and Goldberg, K. (2021, January 23–27). FogROS: An Adaptive Framework for Automating Fog Robotics Deployment. Proceedings of the 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE), Lyon, France.10.1109/CASE49439.2021.9551628 ↗
63. Ou, J., Guo, X., Lou, W., and Zhu, M. (2021). Learning the Spatial Perception and Obstacle Avoidance with the Monocular Vision on a Quadrotor, Institute of Electrical and Electronics Engineers Inc.10.1109/ICMA52036.2021.9512618 ↗
64. Jembre, Y., Nugroho, Y., Khan, M., Attique, M., Paul, R., Shah, S., and Kim, B. (2021). Evaluation of reinforcement and deep learning algorithms in controlling unmanned aerial vehicles. Appl. Sci., 11.10.3390/app11167240 ↗
65. Yuhas, M., Feng, Y., Ng, D., Rahiminasab, Z., and Easwaran, A. (2021). Embedded Out-of-Distribution Detection on an Autonomous Robot Platform, Association for Computing Machinery, Inc.10.1145/3445034.3460509 ↗
66. Timmis, I., Paul, N., and Chung, C.J. (2021, January 14–15). Teaching Vehicles to Steer Themselves with Deep Learning. Proceedings of the 2021 IEEE International Conference on Electro Information Technology (EIT), Mt. Pleasant, MI, USA.10.1109/EIT51626.2021.9491894 ↗
67. Laser measurement key technologies and application in robot autonomous navigation10.1142/S0218001411008920 ↗
68. Unmanned Terrestrial Deep Stereo ConvNet Gofer Embedded with CNN Architecture10.18178/ijmerr.11.11.807-819 ↗
69. HVIOnet: A deep learning based hybrid visual–inertial odometry approach for unmanned aerial system position estimation10.1016/j.neunet.2022.09.001 ↗
70. Zhi, L., and Xuesong, M. (2018, January 12–14). Navigation and Control System of Mobile Robot Based on ROS. Proceedings of the 2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, China.10.1109/IAEAC.2018.8577901 ↗
71. A machine learning based energy efficient trajectory planning approach for industrial robots10.1016/j.procir.2019.03.074 ↗
72. Automatic control of mobile robot based on autonomous navigation algorithm10.1007/s10015-019-00542-0 ↗
73. Deep Reinforcement Learning Based Game-Theoretic Decision-Making for Autonomous Vehicles10.1109/LRA.2021.3134249 ↗
74. Guan, W., and Guo, Y. (2022, January 22–24). A Visual Learning based Robotic Grasping System. Proceedings of the 2022 The 6th International Conference on Advances in Artificial Intelligence, Birmingham, UK.10.1145/3571560.3571564 ↗
75. Learning visual path–following skills for industrial robot using deep reinforcement learning10.1007/s00170-022-09800-1 ↗
76. Su, L., Hua, Y., Dong, X., and Ren, Z. (2022). Human-UAV swarm multi-modal intelligent interaction methods. Hangkong Xuebao/Acta Aeronaut. Astronaut. Sin., 43.
77. Tsai, J., Chang, C.C., Ou, Y.C., Sieh, B.H., and Ooi, Y.M. (2022). Autonomous Driving Control Based on the Perception of a Lidar Sensor and Odometer. Appl. Sci., 12.10.3390/app12157775 ↗
78. Object Detection for Robot Coordination in Robotics Soccer10.4314/njtd.v19i2.5 ↗
79. Khalifa, A., Abdelrahman, A., Strazdas, D., Hintz, J., Hempel, T., and Al-Hamadi, A. (2022). Face Recognition and Tracking Framework for Human–Robot Interaction. Appl. Sci., 12.10.3390/app12115568 ↗
80. Ravi, N., and El-Sharkawy, M. (2022). Real-Time Embedded Implementation of Improved Object Detector for Resource-Constrained Devices. J. Low Power Electron. Appl., 12.10.3390/jlpea12020021 ↗
81. Deep learning based UAV vision object detection and tracking
82. Gong, H., Wang, P., Ni, C., and Cheng, N. (2022). Efficient Path Planning for Mobile Robot Based on Deep Deterministic Policy Gradient. Sensors, 22.10.21203/rs.3.rs-2201974/v1 ↗
83. Baek, E.T., and Im, D.Y. (2022). ROS-Based Unmanned Mobile Robot Platform for Agriculture. Appl. Sci., 12.10.3390/app12094335 ↗
84. Software change-proneness prediction based on deep learning
85. Bui, K., Truong, G., and Ngoc, D. (2022). GCTD3: Modeling of Bipedal Locomotion by Combination of TD3 Algorithms and Graph Convolutional Network. Appl. Sci., 12.10.3390/app12062948 ↗
86. Self-supervised Learning Approach to Local Trajectory Planning for Mobile Robots Using Optimization of Trajectories10.1007/978-981-19-7660-5_66 ↗
87. Tele-operation system for mobile robots using in oil & gas industry; [Sistema de tele-operación para robots móviles en la industria del petróleo y gas]
88. Flexible robotic teleoperation architecture for intelligent oil fields10.1016/j.heliyon.2020.e03833 ↗
89. Mohanty, P.K., Sah, A.K., Kumar, V., and Kundu, S. (2017, January 28). Application of deep Q-learning for wheel mobile robot navigation. Proceedings of the 2017 3rd International Conference on Computational Intelligence and Networks (CINE), Odisha, India.10.1109/CINE.2017.11 ↗
90. de Assis Brasil, P.M., Pereira, F.U., de Souza Leite Cuadros, M.A., Cukla, A.R., and Tello Gamarra, D.F. (2020, January 9–13). A study on global path planners algorithms for the simulated turtlebot 3 robot in ros. Proceedings of the 2020 Latin American Robotics Symposium (LARS), 2020 Brazilian Symposium on Robotics (SBR) and 2020 Workshop on Robotics in Education (WRE), Natal, Brazil.10.1109/LARS/SBR/WRE51543.2020.9307003 ↗
91. The TurtleBot3 Teacher Resources Hands On10.1109/MSPEC.2017.8000281 ↗
92. Amsters, R., and Slaets, P. (2020). Turtlebot 3 as a Robotics Education Platform, Springer.10.1007/978-3-030-26945-6_16 ↗
93. Analysis of the Performance According to Object Density in Static Environments of GA and PSO Algorithms Used in Mobile Robot Path Planning
94. Reviewing Machine Learning Algorithms in the Domain of Healthcare
95. Machine learning and the physical sciences
96. Improving ant colony optimization algorithm with epsilon greedy and Levy flight10.1007/s40747-020-00138-3 ↗
97. Montalvo, W., Escobar-Naranjo, J., Garcia, C.A., and Garcia, M.V. (2020). Low-Cost Automation for Gravity Compensation of Robotic Arm. Appl. Sci., 10.10.3390/app10113823 ↗
98. Ruan, X., Ren, D., Zhu, X., and Huang, J. (2019, January 3–5). Mobile robot navigation based on deep reinforcement learning. Proceedings of the 2019 Chinese Control And Decision Conference (CCDC), Nanchang, China.10.1109/CCDC.2019.8832393 ↗
99. Lin, F., Ji, Z., Wei, C., and Niu, H. (2021). Reinforcement Learning-Based Mapless Navigation with Fail-Safe Localisation, Springer.10.1007/978-3-030-89177-0_10 ↗
100. Tian, S., Lei, S., Huang, Q., and Huang, A. (2022, January 18–21). The application of path planning algorithm based on deep reinforcement learning for mobile robots. Proceedings of the 2022 International Conference on Culture-Oriented Science and Technology (CoST), Lanzhou, China.10.1109/CoST57098.2022.00084 ↗