journal article·2019-11-18Open access

A Novel Searching Method Using Reinforcement Learning Scheme for Multi-UAVs in Unknown Environments

Wei Yue, Xianhe Guan, Liyuan Wang

Applied Sciences· Volume 9· Issue 22

Abstract

In this paper, the important topic of cooperative searches for multi-dynamic targets in unknown sea areas by unmanned aerial vehicles (UAVs) is studied based on a reinforcement learning (RL) algorithm. A novel multi-UAV sea area search map is established, in which models of the environment, UAV dynamics, target dynamics, and sensor detection are involved. Then, the search map is updated and extended using the concept of the territory awareness information map. Finally, according to the search efficiency function, a reward and punishment function is designed, and an RL method is used to generate a multi-UAV cooperative search path online. The simulation results show that the proposed algorithm could effectively perform the search task in the sea area with no prior information.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Distributed Control Multi-Agent Systems · UAV Applications and Optimization · Reinforcement Learning in Robotics

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Marine Electrical Engineering, Dalian Maritime University, Dalian 116000, China

Key Laboratory of Intelligent Perception and Advanced Control of State Ethnic Affairs Commission, Dalian Minzu University, Dalian 116600, China

Funding

the Natural Science Foundation of China · Key Laboratory of Intelligent Perception and Advanced Control of State Ethnic Affairs Commission · the Fundamental Research Funds for the Central Universities

References (21)

Show source-provided references
1. Search and pursuit-evasion in mobile robotics10.1007/s10514-011-9241-4 ↗
2. Modeling and simulation of multi-ship coordinated spiral call-searching
3. Modeling and Simulation of Alpine Leaf search and potential search with Magnetometer
4. A Study of Multi-UAVs Cooperative Area Coverage Search
5. Optimal UAV Route Planning for Coverage Search of Stationary Target in River10.1109/TCST.2017.2781655 ↗
6. Search for targets in a risky environment using multi-objective optimisation10.1049/iet-rsn.2018.5184 ↗
7. A novel coordinated path planning method using k-degree smoothing for multi-UAVs10.1016/j.asoc.2016.06.046 ↗
8. Multiagent Information Fusion and Cooperative Control in Target Search10.1109/TCST.2012.2198650 ↗
9. Coordinated search for a lost target in a Bayesian world10.1163/1568553042674707 ↗
10. Optimal multiplicative Bayesian search for a lost target
11. Addressing Uncertainty in UAV Navigation Decision-Making10.1109/TAES.2008.4517005 ↗
12. Cooperative Search by UAV Teams: A Model Predictive Approach using Dynamic Graphs10.1109/TAES.2011.6034656 ↗
13. Pablo, L.P. (2013). Minimum Time Search of Moving Targets in Uncertain Environments, Universidad Complutense de Madrid.
14. Evolutionary Planning of Multi-UAV Search for Missing Tourists
15. An Information Value Approach to Route Planning for UAV Search and Track Missions10.1109/TAES.2012.6237608 ↗
16. Matsuda, A., Misawa, H., and Horio, K. (2011, January 27–30). Decision making based on reinforcement learning and emotion learning for social behavior. Proceedings of the IEEE International Conference on Fuzzy Systems, Taipei, Taiwan.10.1109/FUZZY.2011.6007506 ↗
17. Cai, Y., Yang, S.X., and Xu, X. (2013, January 16–19). A combined hierarchical reinforcement learning based approach for multi-robot cooperative target searching in complex unknown environments. Proceedings of the 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), Singapore.10.1109/ADPRL.2013.6614989 ↗
18. Pham, H.X., La, H.M., Feil-Seifer, D., and Van Nguyen, L. (2018, January 6–8). Reinforcement Learning for Autonomous UAV Navigation Using Function Approximation. Proceedings of the IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Philadelphia, PA, USA.10.1109/SSRR.2018.8468611 ↗
19. A Q-Learning Approach to Flocking with UAVs in a Stochastic Environment10.1109/TCYB.2015.2509646 ↗
20. Field enclosure experiments on the technique of poisoning the rabbit, Oryctolagus cuniculus (L.). II. A study of territorial behaviour and the use of bait stations10.1071/CWR9630028 ↗
21. Balancing search and target response in cooperative unmanned aerial vehicle (UAV) teams10.1109/TSMCB.2005.861881 ↗