journal article·2025Open access

Research on Adaptive Reward Optimization Method for Robot Navigation in Complex Dynamic Environment

Jie He, Dongmei Zhao, Tao Liu, Qingfeng Zou, Jian’an Xie

Computers, Materials & Continua· Volume 84· Issue 2

Abstract

Robot navigation in complex crowd service scenarios, such as medical logistics and commercial guidance, requires a dynamic balance between safety and efficiency, while the traditional fixed reward mechanism lacks environmental adaptability and struggles to adapt to the variability of crowd density and pedestrian motion patterns. This paper proposes a navigation method that integrates spatiotemporal risk field modeling and adaptive reward optimization, aiming to improve the robot’s decision-making ability in diverse crowd scenarios through dynamic risk assessment and nonlinear weight adjustment. We construct a spatiotemporal risk field model based on a Gaussian kernel function by combining crowd density, relative distance, and motion speed to quantify environmental complexity and realize crowd-density-sensitive risk assessment dynamically. We apply an exponential decay function to reward design to address the linear conflict problem of fixed weights in multi-objective optimization. We adaptively adjust weight allocation between safety constraints and navigation efficiency based on real-time risk values, prioritizing safety in highly dense areas and navigation efficiency in sparse areas. Experimental results show that our method improves the navigation success rate by 9.0% over state-of-the-art models in high-density scenarios, with a 10.7% reduction in intrusion time ratio. Simulation comparisons validate the risk field model’s ability to capture risk superposition effects in dense scenarios and the suppression of near-field dangerous behaviors by the exponential decay mechanism. Our parametric optimization paradigm establishes an explicit mapping between navigation objectives and risk parameters through rigorous mathematical formalization, providing an interpretable approach for safe deployment of service robots in dynamic environments.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Robotic Path Planning Algorithms

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (29)

Show source-provided references
1. Planning and control of autonomous mobile robots for intralogistics: literature review and research agenda10.1016/j.ejor.2021.01.019 ↗
2. Preference for robot service or human service in hotels? impacts of the COVID-19 pandemic10.1016/j.ijhm.2020.102795 ↗
3. A comprehensive review of mobile robot navigation using deep reinforcement learning algorithms in crowded environments10.1007/s10846-024-02198-w ↗
4. A review of brain-inspired cognition and navigation technology for mobile robots10.34133/cbsystems.0128 ↗
5. Safe and socially compliant robot navigation in crowds with fast-moving pedestrians via deep reinforcement learning10.1017/S0263574724000183 ↗
6. Decentralized structural-RNN for robot crowd navigation with deep reinforcement learning
7. Robot navigation in a crowd by integrating deep reinforcement learning and online planning10.1007/s10489-022-03191-2 ↗
8. Risk-aware deep reinforcement learning for robot crowd navigation10.3390/electronics12234744 ↗
9. Intention aware robot crowd navigation with attention-based interaction graph
10. Belief aided navigation using Bayesian reinforcement learning for avoiding humans in blind spots
11. Transformable gaussian reward function for socially aware navigation using deep reinforcement learning10.3390/s24144540 ↗
12. Crowd-robot interaction: crowd-aware robot navigation with attention-based deep reinforcement learning
13. A formal basis for the heuristic determination of minimum cost paths10.1109/TSSC.1968.300136 ↗
14. Sotirchos G, Ajanovic Z. Search-based versus sampling-based robot motion planning: a comparative study. arXiv:240609623. 2024.
15. Van Den Berg · 2009 Aug 31–Sep 3
16. Robot navigation in crowds by graph convolutional networks with attention learned from human gaze10.1109/LRA.2020.2972868 ↗
17. Learning crowd behaviors in navigation with attention-based spatial-temporal graphs
18. Deep reinforcement learning based mobile robot navigation: a review10.26599/TST.2021.9010012 ↗
19. Comprehensive overview of reward engineering and shaping in advancing reinforcement learning applications10.1109/ACCESS.2024.3504735 ↗
20. Dynamic warning zone and a short-distance goal for autonomous robot navigation using deep reinforcement learning10.1007/s40747-023-01216-y ↗
21. DWA-RL: dynamically feasible deep reinforcement learning policy for robot navigation among mobile obstacles
22. Scan: socially-aware navigation using Monte Carlo tree search
23. Deep reinforcement learning for active target tracking
24. Memory-based crowd-aware robot navigation using deep reinforcement learning10.1007/s40747-022-00906-3 ↗
25. From proxemics theory to socially-aware navigation: a survey10.1007/s12369-014-0251-1 ↗
26. Goyal P, Niekum S, Mooney RJ. Using natural language for reward shaping in reinforcement learning. arXiv:190302020. 2019.10.24963/ijcai.2019/331 ↗
27. SafeCrowdNav: safety evaluation of robot crowd navigation in complex scenes10.3389/fnbot.2023.1276519 ↗
28. Research on autonomous navigation and control algorithm of intelligent robot based on reinforcement learning10.12694/scpe.v26i1.3841 ↗
29. Deep learning-based vision systems for robot semantic navigation: an experimental study10.3390/technologies12090157 ↗