A Centralized–Distributed Joint Routing Algorithm for LEO Satellite Constellations Based on Multi-Agent Reinforcement Learning
Source abstract
Abstract
Designing routing algorithms for Low Earth Orbit (LEO) satellite networks poses a significant challenge due to their high dynamics, frequent link failures, and unevenly distributed traffic. Existing studies predominantly focus on shortest-path solutions, which compute minimum-delay paths using global topology information but often neglect the impact of traffic load on routing performance and struggle to adapt to rapid link-state variations. In this regard, we propose a Multi-Agent Reinforcement Learning-Based Joint Routing (MARL-JR) algorithm, which integrates centralized and distributed routing algorithms. MARL-JR combines the accuracy of centralized methods with the responsiveness of distributed approaches in handling dynamic disruptions. In MARL-JR, ground stations initialize Q-tables and upload them to satellites, reducing onboard computational overhead while enhancing routing performance. Compared to traditional centralized algorithms, MARL-JR achieves faster link-state awareness and adaptation; compared to distributed algorithms, it delivers superior initial performance due to optimized pre-training. Experimental results demonstrate that MARL-JR outperforms both Q-Routing (QR) and DR-BM algorithms in average delay, packet loss rate, and load-balancing efficiency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Satellite Communication Systems · Opportunistic and Delay-Tolerant Networks · Mobile Agent-Based Network Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China
Innovation Academy for Microsatellites of CAS, Shanghai 201304, China
References (27)
Show source-provided references
1. Tsai, K.-C., Fan, L., Wang, L.-C., Lent, R., and Han, Z. (2022, January 10–13). Multi-Commodity Flow Routing for Large-Scale LEO Satellite Networks Using Deep Reinforcement Learning. Proceedings of the 2022 IEEE Wireless Communications and Networking Conference (WCNC), Austin, TX, USA.10.1109/WCNC51071.2022.9771734 ↗
2. Broadband LEO Satellite Communications: Architectures and Key Technologies10.1109/MWC.2019.1800299 ↗
3. Lei, Y.H., Cao, L.F., and Han, M.D. (2021, January 10–13). A Handover Strategy Based on User Dynamic Preference for LEO Satellite. Proceedings of the 2021 7th International Conference on Computer and Communications (ICCC), Chengdu, China.10.1109/ICCC54389.2021.9674431 ↗
4. A dynamic routing concept for ATM-based satellite personal communication networks10.1109/49.634801 ↗
5. Gounder, V.V., Prakash, R., and Abu-Amara, H. (1999, January 12–13). Routing in LEO-based satellite networks. Proceedings of the 1999 IEEE Emerging Technologies Symposium Wireless Communications and Systems, Richardson, TX, USA.
6. QoS guarantees for multimedia services on a TDMA-based satellite network10.1109/35.601743 ↗
7. Temporal Netgrid Model-Based Dynamic Routing in Large-Scale Small Satellite Networks10.1109/TVT.2019.2910570 ↗
8. Wang, X., Dai, Z., and Xu, Z. (2021, January 7–10). LEO Satellite Network Routing Algorithm Based on Reinforcement Learning. Proceedings of the 2021 IEEE 4th International Conference on Electronics Technology (ICET), Chengdu, China.10.1109/ICET51757.2021.9451072 ↗
9. Deep Reinforcement Learning-Based Routing and Spectrum Assignment of EONs by Exploiting GCN and RNN for Feature Extraction10.1109/JLT.2022.3175865 ↗
10. Reinforcement learning based dynamic distributed routing scheme for mega LEO satellite networks10.1016/j.cja.2022.06.021 ↗
11. Wang, C., Wang, H., and Wang, W. (2019). A two-hops state-aware routing strategy based on deep reinforcement learning for LEO satellite networks. Electronics, 8.10.3390/electronics8090920 ↗
12. Zuo, P., Wang, C., Yao, Z., Hou, S., and Jiang, H. (2021, January 27–30). An intelligent routing algorithm for LEO satellites based on deep reinforcement learning. Proceedings of the 2021 IEEE 94th Vehicular Technology Conference (VTC2021-Fall), Norman, OK, USA.10.1109/VTC2021-Fall52928.2021.9625325 ↗
13. Shi, Y., Yuan, Z., Zhu, X., and Zhu, H. (2023). An Adaptive Routing Algorithm for Inter-Satellite Networks Based on the Combination of Multipath Transmission and Q-Learning. Processes, 11.10.3390/pr11010167 ↗
14. Hybrid constellation design using a genetic algorithm for a leo-based navigation augmentation system10.1007/s10291-020-00977-0 ↗
15. Leo satellite constellation for internet of things10.1109/ACCESS.2017.2735988 ↗
16. Intersatellite communication system based on visible light10.1109/TAES.2018.2832938 ↗
17. Handley, M. (2019, January 13–15). Using ground relays for low-latency wide-area routing in megaconstellations. Proceedings of the 18th ACM Workshop on Hot Topics in Networks, Princeton, NJ, USA.10.1145/3365609.3365859 ↗
18. The Low Earth Orbit Satellite Population and Impacts of the SpaceX Starlink Constellation10.3847/2041-8213/ab8016 ↗
19. Pelton, J.N., and Madry, S. (2020). The OneWeb Satellite System. Handbook of Small Satellites, Springer.10.1007/978-3-030-20707-6 ↗
20. Delay performance comparison across seven Low-Earth-Orbit (LEO) satellite constellations
21. Chang, H.S., Kim, B.W., Lee, C.G., Choi, Y., Min, S.L., Yang, H.S., and Kim, C.S. (1995, January 14–16). Topological design and routing for low-Earth orbit satellite networks. Proceedings of the GLOBECOM’95, Singapore.
22. Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., and Mordatch, I. (2017, January 4–9). Multi-agent actor-critic for mixed cooperative-competitive environments. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
23. QGeo: Q-Learning-Based Geographic Ad Hoc Routing Protocol for Unmanned Robotic Networks10.1109/LCOMM.2017.2656879 ↗
24. Multipath Cooperative Routing with Efficient Acknowledgement for LEO Satellite Networks10.1109/TMC.2018.2831679 ↗
25. A note on two problems in connexion with graphs10.1007/BF01386390 ↗
26. Soret, B., Leyva-Mayorga, I., Lozano-Cuadra, F., and Thorsager, M.D. (2024, January 5–8). Q-learning for distributed routing in LEO satellite constellations. Proceedings of the 2024 IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN), Stockholm, Sweden.10.1109/ICMLCN59089.2024.10624807 ↗
27. Boyan, J.A., and Littman, M. (1993). Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach. Advances in Neural Information Processing Systems, Proceedings of the 7th International Conference on Neural Information Processing Systems, Denver, CO, USA, 29 November–2 December 1993, Morgan Kaufmann Publishers Inc.