Group Attention Aware Coordination Graph
Source abstract
Abstract
Cooperative Multi-Agent Reinforcement Learning (MARL) relies on effective coordination among agents to maximize team performance in complex environments. However, existing coordination graph-based approaches often overlook dynamic group structures and struggle to accurately capture fine-grained inter-agent dependencies. In this paper, we introduce a novel method called the Group Attention Aware Coordination Graph (G2ACG), which builds upon the group modeling capabilities of the Group-Aware Coordination Graph (GACG). G2ACG incorporates a dynamic attention mechanism to dynamically compute edge weights in the coordination graph, enabling a more flexible and fine-grained representation of agent interactions. These learned edge weights guide a Graph Attention Network (GAT) to perform message passing and representation learning, and the resulting features are integrated into a global policy via QMIX for cooperative decision-making. Experimental results on the StarCraft II Multi-Agent Challenge (SMAC) benchmark show that G2ACG consistently outperforms strong baselines, including QMIX, DICG, and GACG, across various scenarios with diverse agent types and population sizes. Ablation studies further confirm the effectiveness of the proposed attention mechanism, demonstrating that both the number of attention heads and the number of GAT layers significantly affect performance, with a two-layer GAT and multi-head attention configuration yielding the best results.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Distributed Control Multi-Agent Systems · Advanced Graph Neural Networks · Neural Networks and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Intelligence Science and Technology, National University of Defence Technology, Changsha 410073, China
References (20)
Show source-provided references
1. Zhang, K., Yang, Z., and Başar, T. (2021). Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, Springer.10.1007/978-3-030-60990-0_12 ↗
2. Group-Aware Coordination Graph for Multi-Agent Reinforcement Learning
3. Sunehag, P., Lever, G., Gruslys, A., Czarnecki, W.M., Zambaldi, V., Jaderberg, M., Lanctot, M., Sonnerat, N., Leibo, J.Z., and Tuyls, K. (2018, January 10–15). Value-Decomposition Networks for Cooperative Multi-Agent Learning Based on Team Reward. Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS ’18, Stockholm, Sweden.
4. Monotonic value function factorisation for deep multi-agent reinforcement learning
5. Böhmer, W., Kurin, V., and Whiteson, S. (2020, January 12–18). Deep coordination graphs. Proceedings of the 37th International Conference on Machine Learning, Vienna, Austria.
6. Gupta, N., Hare, J.Z., Kannan, R., and Prasanna, V. (2025). Deep Meta Coordination Graphs for Multi-agent Reinforcement Learning. arXiv.
7. Brody, S., Alon, U., and Yahav, E. (2021). How attentive are graph attention networks?. arXiv.
8. Luo, B., Cheng, L., Wu, Z.G., Li, H., and Li, C. (2023, January 20–23). Mastering Complex Coordination Through Attention-Based Dynamic Graph. Proceedings of the Neural Information Processing, Changsha, China.
9. Liu, Y., Wang, W., Hu, Y., Hao, J., Chen, X., and Gao, Y. (2020, January 7–12). Multi-Agent Game Abstraction via Graph Attention Neural Network. Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA.
10. Samvelyan, M., Rashid, T., Schroeder de Witt, C., Farquhar, G., Nardelli, N., Rudner, T.G.J., Hung, C.M., Torr, P.H.S., Foerster, J., and Whiteson, S. (2019, January 13–17). The StarCraft Multi-Agent Challenge. Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS ’19, Montreal, QC, Canada.
11. Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (May, January 30). Graph attention networks. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
12. Collaborative Multiagent Reinforcement Learning by Payoff Propagation
13. Evolutionary Dynamic Optimization: A Survey of the State of the art10.1016/j.swevo.2012.05.001 ↗
14. Li, S., Gupta, J.K., Morales, P., Allen, R., and Kochenderfer, M.J. (2021, January 3–7). Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learning. Proceedings of the Adaptive Agents and Multi-Agent Systems, Virtual.
15. Ruan, J., Du, Y., Xiong, X., Xing, D., Li, X., Meng, L., Zhang, H., Wang, J., and Xu, B. (2022, January 9–13). GCS: Graph-Based Coordination Strategy for Multi-Agent Reinforcement Learning. Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’22, Virtual.
16. Inferring Latent Temporal Sparse Coordination Graph for Multiagent Reinforcement Learning10.1109/TNNLS.2024.3513402 ↗
17. Finkelshtein, B., Huang, X., Bronstein, M.M., and Ceylan, I.I. (2023). Cooperative Graph Neural Networks. arXiv.
18. Goeckner, A., Sui, Y., Martinet, N., Li, X., and Zhu, Q. (2024, January 14–18). Graph Neural Network-based Multi-agent Reinforcement Learning for Resilient Distributed Coordination of Multi-Robot Systems. Proceedings of the 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Abu Dhabi, United Arab Emirates.10.1109/IROS58592.2024.10802510 ↗
19. Oliehoek, F.A., and Amato, C. (2016). A Concise Introduction to Decentralized POMDPs, Springer.10.1007/978-3-319-28929-8 ↗
20. Kipf, T.N., and Welling, M. (2017, January 24–26). Semi-Supervised Classification with Graph Convolutional Networks. Proceedings of the 5th International Conference on Learning Representations, ICLR 2017, Toulon, France. Available online: https://OpenReview.net.