QMIX-GNN: A Graph Neural Network-Based Heterogeneous Multi-Agent Reinforcement Learning Model for Improved Collaboration and Decision-Making
Source abstract
Abstract
In multi-agent reinforcement learning, the fully centralized approach suffers from issues such as explosion of the joint state and action spaces, leading to performance degradation. On the other hand, the fully decentralized approach relies on agents that focus solely on maximizing their own rewards, making effective collaboration difficult and complicating adaption to scenarios that require cooperation among multiple agents. The Centralized Training and Decentralized Execution (CTDE) framework combines both fully centralized and fully decentralized approaches. During the training phase, a virtual central node receives the observations and actions of all agents for training, while during the execution phase each agent makes decisions based only on its own observations. However, in this framework the agents do not fully consider the information of other agents or the complex interactions between them during execution, which affects the correctness of their decisions. Therefore, this paper proposes a heterogeneous multi-agent reinforcement learning model based on graph neural networks, which we call QMIX-GNN. This model efficiently and flexibly handles input data of different dimensions, enabling the fusion of heterogeneous multi-agent information and providing this fused information to the agents. In turn, this allows them to perceive more comprehensive information and improve the correctness of their decisions. Experimental results demonstrate that the QMIX-GNN model performs better than other methods on complex multi-agent collaborative tasks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Reinforcement Learning in Robotics · Collaboration in agile enterprises
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Laboratory of Intelligent Collaborative Computing, School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
Funding
Chengdu Key Research and Development Support Program “Jie Bang Gua Shuai” Project
References (27)
Show source-provided references
1. A survey on multi-agent reinforcement learning and its application
2. Multi-agent deep reinforcement learning: A survey10.1007/s10462-021-09996-w ↗
3. Ruder, S. (2017). An Overview of Multi-Task Learning in Deep Neural Networks. arXiv.
4. VSP Upgoing and Downgoing Wavefield Separation: A Hybrid Model-Data Driven Approach10.1109/TGRS.2025.3547941 ↗
5. Qian, F., Pan, S., and Zhang, G. (2025). Tensor Computation for Seismic Data Processing: Linking Theory and Practice, Springer.
6. Kong, G., Chen, F., Yang, X., Cheng, G., Zhang, S., and He, W. (2024). Optimal Deception Asset Deployment in Cybersecurity: A Nash Q-Learning Approach in Multi-Agent Stochastic Games. Appl. Sci., 14.10.3390/app14010357 ↗
7. Wilson, A., Menzies, R., Morarji, N., Foster, D., Mont, M.C., Turkbeyler, E., and Gralewski, L. (2024). Multi-Agent Reinforcement Learning for Maritime Operational Technology Cyber Security. arXiv.
8. Du, J., Yu, A., Zhou, H., Jiang, Q., and Bai, X. (2025). Research on Integrated Control Strategy for Highway Merging Bottlenecks Based on Collaborative Multi-Agent Reinforcement Learning. Appl. Sci., 15.10.3390/app15020836 ↗
9. Garcia-Cantón, S., Ruiz de Mendoza, C., Cervelló-Pastor, C., and Sallent, S. (2025). Multi-Agent Reinforcement Learning-Based Routing and Scheduling Models in Time-Sensitive Networking for Internet of Vehicles Communications Between Transportation Field Cabinets. Appl. Sci., 15.10.3390/app15031122 ↗
10. Mamond, A.W., Kundroo, M., Yoo, S.e., Kim, S., and Kim, T. (2025). FLDQN: Cooperative Multi-Agent Federated Reinforcement Learning for Solving Travel Time Minimization Problems in Dynamic Environments Using SUMO Simulation. Sensors, 25.10.3390/s25030911 ↗
11. Zhang, C., Zhou, X., Xu, C., Wu, Z., Liu, J., and Qi, H. (2025). Automatic Generation of Precast Concrete Component Fabrication Drawings Based on BIM and Multi-Agent Reinforcement Learning. Buildings, 15.10.3390/buildings15020284 ↗
12. Survey of recent multi-agent reinforcement learning algorithms utilizing centralized training
13. Xiao, L., Wu, X., Wu, W., Yang, J., and He, L. (2022, January 23–27). Multi-channel attentive graph convolutional network with sentiment fusion for multimodal sentiment analysis. Proceedings of the ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9747542 ↗
14. A comprehensive survey on graph neural networks10.1109/TNNLS.2020.2978386 ↗
15. Graph Neural Network Meets Multi-Agent Reinforcement Learning: Fundamentals, Applications, and Future Directions10.1109/MWC.015.2300595 ↗
16. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. (May, January 30). Graph Attention Networks. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
17. Ding, S., Du, W., Ding, L., Zhang, J., Guo, L., and An, B. (2023). Multiagent Reinforcement Learning With Graphical Mutual Information Maximization. IEEE Trans. Neural Netw. Learn. Syst., 1–10.10.1109/TNNLS.2023.3243557 ↗
18. Graph neural network and multi-agent reinforcement learning for machine-process-system integrated control to optimize production yield10.1016/j.jmsy.2022.05.018 ↗
19. Leibfried, F., and Grau-Moya, J. (2020, January 9–12). Mutual-information regularization in Markov decision processes and actor-critic learning. Proceedings of the Conference on Robot Learning, Graz, Austria.
20. Oliehoek, F.A., and Amato, C.A. (2016). A Concise Introduction to Decentralized POMDPs, Springer.10.1007/978-3-319-28929-8 ↗
21. Lowe, R., Wu, Y.I., Tamar, A., Harb, J., Pieter Abbeel, O., and Mordatch, I. (2017, January 4–9). Multi-agent actor-critic for mixed cooperative-competitive environments. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA.
22. Monotonic value function factorisation for deep multi-agent reinforcement learning
23. Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., and Whiteson, S. (2018, January 2–7). Counterfactual multi-agent policy gradients. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11794 ↗
24. Son, K., Kim, D., Kang, W.J., Hostallero, D.E., and Yi, Y. (2019, January 9–15). Qtran: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA.
25. Kipf, T.N., and Welling, M. (2017, January 24–26). Semi-Supervised Classification with Graph Convolutional Networks. Proceedings of the International Conference on Learning Representations (ICLR), Toulon, France.
26. A survey of multi-agent deep reinforcement learning with communication10.1007/s10458-023-09633-6 ↗
27. Samvelyan, M., Rashid, T., De Witt, C.S., Farquhar, G., Nardelli, N., Rudner, T.G., Hung, C.M., Torr, P.H., Foerster, J., and Whiteson, S. (2019). The starcraft multi-agent challenge. arXiv.