Multi-Agent Reinforcement Learning in Games: Research and Applications
Source abstract
Abstract
Biological systems, ranging from ant colonies to neural ecosystems, exhibit remarkable self-organizing intelligence. Inspired by these phenomena, this study investigates how bio-inspired computing principles can bridge game-theoretic rationality and multi-agent adaptability. This study systematically reviews the convergence of multi-agent reinforcement learning (MARL) and game theory, elucidating the innovative potential of this integrated paradigm for collective intelligent decision-making in dynamic open environments. Building upon stochastic game and extensive-form game-theoretic frameworks, we establish a methodological taxonomy across three dimensions: value function optimization, policy gradient learning, and online search planning, thereby clarifying the evolutionary logic and innovation trajectories of algorithmic advancements. Focusing on complex smart city scenarios—including intelligent transportation coordination and UAV swarm scheduling—we identify technical breakthroughs in MARL applications for policy space modeling and distributed decision optimization. By incorporating bio-inspired optimization approaches, the investigation particularly highlights evolutionary computation mechanisms for dynamic strategy generation in search planning, alongside population-based learning paradigms for enhancing exploration efficiency in policy refinement. The findings reveal core principles governing how groups make optimal choices in complex environments while mapping the technological development pathways created by blending cross-disciplinary methods to enhance multi-agent systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Reinforcement Learning in Robotics · Adaptive Dynamic Programming Control · Traffic control and management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
High-Tech Institute of Xi’an, Xi’an 710038, China
School of Computer Science and Technology, Zhejiang Normal University, Jinhua 321004, China
References (78)
Show source-provided references
1. Multi-Agent Deep Reinforcement Learning for Recharging-Considered Vehicle Scheduling Problem in Container Terminals10.1109/TITS.2024.3412932 ↗
2. Wang, K., Shen, Z., Lei, Z., Liu, X., and Zhang, T. (2024). Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs. IEEE Transactions on Mobile Computing, IEEE.10.1109/TMC.2025.3556243 ↗
3. Distributed real-time scheduling in cloud manufacturing by deep reinforcement learning10.1109/TII.2022.3178410 ↗
4. Community Microgrid Energy Co-Scheduling Based on Deep Reinforcement Learning and Contribution Mechanisms10.1109/TSG.2024.3461320 ↗
5. A multi-agent path planning algorithm based on game theory and reinforcement learning10.3724/SP.J.1249.2024.03274 ↗
6. Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learning10.1109/TITS.2024.3352446 ↗
7. Qian, T., and Yang, C. (2024). Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment. Front. Energy Res., 11.10.3389/fenrg.2023.1333827 ↗
8. Nash Q-Learning for General-Sum Stochastic Games
9. Learning Rate Curriculum10.1007/s11263-024-02186-5 ↗
10. Automatically designing counterfactual regret minimization algorithms for solving imperfect information games10.1016/j.artint.2024.104232 ↗
11. Hierarchical multi-agent deep reinforcement learning with adjustable hierarchy for home energy management systems10.1016/j.enbuild.2025.115391 ↗
12. Intelligent games meeting with multi-agent deep reinforcement learning: A comprehensive review10.1007/s10462-025-11166-1 ↗
13. Corrigendum to on the complexity of computing Markov perfect equilibrium in general-sum stochastic games10.1093/nsr/nwad024 ↗
14. A Variant of the Logistic Quantal Response Equilibrium to Select a Perfect Equilibrium10.1007/s10957-024-02433-2 ↗
15. Korivand, S., Galvani, G., Ajoudani, A., Gong, J., and Jalili, N. (2024). Optimizing Human-Robot Teaming Performance through Q-Learning-Based Task Load Adjustment and Physiological Data Analysis. Sensors, 24.10.3390/s24092817 ↗
16. Distributed Q-Learning-Based Voltage Restoration Algorithm in Isolated AC Microgrids Subject to Input Saturation10.1109/TIA.2024.3379965 ↗
17. Action-Project Method: An approach to describing and studying goal-oriented joint actions10.1111/medu.14935 ↗
18. Park, Y.-J., Kim, J.-E., Lee, S.-H., and Cho, K.-H. (2022). An Effective Design Formula for Single-Layer Printed Spiral Coils with the Maximum Quality Factor (Q-Factor) in the Megahertz Frequency Range. Sensors, 22.10.3390/s22207761 ↗
19. Minimax Q-learning design for H ∞ control of linear discrete-time systems10.1631/FITEE.2000446 ↗
20. A large-scale equilibrium model of energy emergency production: Embedding social choice rules into Nash Q-learning automatically achieving consensus of urgent recovery behaviors10.1016/j.energy.2022.125023 ↗
21. Zhao, X., Hu, H., and Sun, D. (2024). Cooperation with Humans of Unknown Intentions in Confined Spaces Using the Stackelberg Friend-or-Foe Game. IEEE Transactions on Aerospace and Electronic Systems, IEEE.10.1109/TAES.2024.3521935 ↗
22. A novel multi-agent decentralized win or learn fast policy hill-climbing with eligibility trace algorithm for smart generation control of interconnected complex power grids10.1016/j.enconman.2015.06.030 ↗
23. Rationality of Learning Algorithms in Repeated Normal-Form Games10.1109/LCSYS.2024.3486631 ↗
24. A Sequential Game Framework for Target Tracking10.1109/ACCESS.2024.3515657 ↗
25. Optimal resource allocation in hierarchical organizations under uncertainty: An interval-valued n-person cooperative game approach
26. FM3Q: Factorized Multi-Agent MiniMax Q-Learning for Two-Team Zero-Sum Markov Game10.1109/TETCI.2024.3383454 ↗
27. A Generalized Minimax Q-learning Algorithm for Two-Player Zero-Sum Stochastic Games10.1109/TAC.2022.3159453 ↗
28. Reinforcement Learning: Computing the Temporal Difference of Values via Distinct Corticostriatal Pathways: (Trends in Neurosciences 35, 457–467; 2012)10.1016/j.tins.2017.05.006 ↗
29. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
30. Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12–17). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI 2016), Phoenix, AZ, USA.10.1609/aaai.v30i1.10295 ↗
31. Wang, L., Wang, Y., and Zheng, B. (2023, January 14–16). Dynamic Power Control Method Based on Stacked SRU Network Combined with NoisyNet DQN for CRN. Proceedings of the 2023 19th International Conference on Mobility, Sensing and Networking (MSN), Nanjing, China.10.1109/MSN60784.2023.00028 ↗
32. Dual-Correction–Adaptation Network for Noisy Knowledge Transfer10.1109/TNNLS.2023.3322390 ↗
33. Optimization of End-to-End AoI in Edge-Enabled Vehicular Fog Systems: A Dueling-DQN Approach
34. Deguale, D.A., Yu, L., Sinishaw, M.L., and Li, K. (2024). Enhancing Stability and Performance in Mobile Robot Path Planning with PMR-Dueling DQN Algorithm. Sensors, 24.10.3390/s24051523 ↗
35. Kamran, D., Engelgeh, T., Busch, M., Fischer, J., and Stiller, C. (2021). Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning. arXiv.10.1109/IROS51168.2021.9636847 ↗
36. Path planning of the unmanned surface vehicle in island and reef environments with time-varying vortices10.1016/j.oceaneng.2024.120231 ↗
37. Attention-Based Distributional Reinforcement Learning for Safe and Efficient Autonomous Driving10.1109/LRA.2024.3427551 ↗
38. Nachum, O., Norouzi, M., Xu, K., and Schuurmans, D. (2017). Bridging the Gap Between Value and Policy Based Reinforcement Learning. arXiv.
39. Chrustowski, K., and Duch, P. (2025). Policy-Based Reinforcement Learning Approach in Imperfect Information Card Game. Appl. Sci., 15.10.3390/app15042121 ↗
40. Lee, K.M., Subramanian, S.G., and Crowley, M. (2022). Investigation of independent reinforcement learning algorithms in multi-agent environments. Front. Artif. Intell., 5.10.3389/frai.2022.805823 ↗
41. Deep Reinforcement Learning-Based Pedestrian and Independent Vehicle Safety Fortification Using Intelligent Perception10.4018/IJSSCI.291712 ↗
42. Independent Deep Deterministic Policy Gradient Reinforcement Learning in Cooperative Multiagent Pursuit Games
43. Programming backgammon using self-teaching neural nets10.1016/S0004-3702(01)00110-2 ↗
44. Tandon, A., and Karlapalem, K. (2018, January 13–19). Medusa: Towards Simulating a Multi-Agent Hide-and-Seek Game. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, Stockholm, Sweden.10.24963/ijcai.2018/866 ↗
45. Wang, K., Tang, Z., and Mu, C. (2024). Dynamic Event-Triggered Model-Free Reinforcement Learning for Cooperative Control of Multiagent Systems. IEEE Transactions on Reliability, IEEE.10.1109/TR.2024.3485211 ↗
46. Self-play training for general impulsive orbital game10.1088/1742-6596/2977/1/012106 ↗
47. Neural Fictitious Self-Play for Radar Anti-Jamming Dynamic Game with Imperfect In-formation10.1109/TAES.2022.3175186 ↗
48. Heinrich, J., and Silver, D. (2016). Deep Reinforcement Learning from Self-Play in Imperfect-Information Games. arXiv.
49. A novel policy gradient algorithm with PSO-based parameter exploration for continuous control10.1016/j.engappai.2020.103525 ↗
50. Learning Adaptive Differential Evolution by Natural Evolution Strategies10.1109/TETCI.2022.3210927 ↗
51. Neighbourhood-based small-world network differential evolution with novelty search strategy10.1504/IJBIC.2023.134971 ↗
52. A Population-based Hybrid Approach for Hyperparameter Optimization of Neural Networks10.1109/ACCESS.2023.3277310 ↗
53. Proximal evolutionary strategy: Improving deep reinforcement learning through evolutionary policy optimization10.1007/s12293-024-00419-1 ↗
54. Jaderberg, M., Dalibard, V., Osindero, S., Czarnecki, W.M., Donahue, J., Razavi, A., Vinyals, O., Green, T., Dunning, I., and Simonyan, K. (2017). Population based training of neural networks. arXiv.
55. Generalized Population-Based Training for Hyperparameter Optimization in Reinforcement Learning10.1109/TETCI.2024.3389777 ↗
56. MALib: A parallel framework for population-based multi-agent reinforcement learning
57. A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat10.1109/TSMC.2023.3270444 ↗
58. Energy management of solar spectral beam-splitting integrated energy systems using Soft Actor-Critic Method10.1016/j.applthermaleng.2025.125966 ↗
59. Energy optimization management of microgrid using improved soft actor-critic algorithm10.61435/ijred.2024.59988 ↗
60. Energy management in HDHEV with dual APUs: Enhancing soft actor-critic using clustered experience replay and multi-dimensional priority sampling10.1016/j.energy.2025.134926 ↗
61. Study on indoor temperature optimal control of air-conditioning based on Twin Delayed Deep Deterministic policy gradient algorithm10.1016/j.enbuild.2024.114420 ↗
62. Mastering the game of Go with deep neural networks and tree search10.1038/nature16961 ↗
63. Post-storm repair crew dispatch for distribution grid restoration using stochastic Monte Carlo tree search and deep neural networks10.1016/j.ijepes.2022.108477 ↗
64. Adaptive Hierarchical Text Classification Using ERNIE and Dynamic Threshold Pruning10.1109/ACCESS.2024.3519954 ↗
65. Dynamic UCB Adaptation Strategy for MCTS based GVGAI
66. Perez, D., Samothrakis, S., Lucas, S., and Rohlfshagen, P. (2013, January 6–10). Rolling Horizon Evolution versus Tree Search for Navigation in Single-Player Real-Time Games. Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation (GECCO’13), New York, NY, USA.10.1145/2463372.2463413 ↗
67. Enhanced Rolling Horizon Evolution Algorithm with Opponent Model Learning: Results for the Fighting Game AI Competition10.1109/TG.2020.3022698 ↗
68. A survey on deep reinforcement learning approaches for traffic signal control10.1016/j.engappai.2024.108100 ↗
69. Yu, C., Velu, A., Vinitsky, E., Gao, J., Wang, Y., Bayen, A., and Wu, Y. (2021). The Surprising Effectiveness of MAPPO in Cooperative, Multi-Agent Games. arXiv.
70. HORSE-CFR: Hierarchical opponent reasoning for safe exploitation counter-factual regret minimization10.1016/j.eswa.2024.125697 ↗
71. POMDP Motion Planning Algorithm Based on Multi-Modal Driving Intention10.1109/TIV.2022.3209926 ↗
72. Safe option-critic: Learning safety in the option-critic architecture10.1017/S0269888921000035 ↗
73. Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., Hashme, S., and Hesse, C. (2019). Dota 2 with Large Scale Deep Reinforcement Learning. arXiv.
74. Espeholt, L., Marinier, R., Stanczyk, P., Wang, K., and Michalski, M. (2019). SEED RL: Scalable and Efficient Deep-RL with Accelerated Central Inference. arXiv.
75. Crop Land Change Detection with MC&N-PSPNet10.3390/app14135429 ↗
76. Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10–15). Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. Proceedings of the Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden.
77. Christodoulou, P. (2019). Soft Actor-Critic for Discrete Action Settings. arXiv.
78. Beyond Nash Equilibrium: Solution Concepts for the 21st Century