A Supply Chain Inventory Management Method for Civil Aircraft Manufacturing Based on Multi-Agent Reinforcement Learning
Source abstract
Abstract
Effective supply chain inventory management is crucial for large-scale manufacturing industries such as civil aircraft and automobile manufacturing to ensure efficient manufacturing. Generally, the main manufacturer makes the annual inventory management plan, and contacts with suppliers when some material is approaching critical inventory level according to the actual production schedule, which increases the difficulty of inventory management. In recent years, many researchers have focused on using reinforcement learning method to study inventory management problems. Current approaches were mainly designed for the supply chain with single-node multi-material or multi-node single-material mode, which are not suitable to the civil aircraft manufacturing supply chain with multi-node multi-material mode. To deal with this problem, we formulated the problem as a partially observable Markov decision process (POMDP) model and proposed a multi-agent reinforcement learning method for supply chain inventory management, in which the dual-policy and information transmission mechanism was designed to help the supply chain participant improve the global information utilization efficiency of the supply chain and the coordination efficiency with other participants. The experiment results show that our method has about 45% performance improvement on efficiency compared with current reinforcement learning-based methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Supply Chain and Inventory Management · Sustainable Supply Chain Management · Supply Chain Resilience and Risk Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Electronics and Information Engineering, Tongji University, Shanghai 201804, China
COMAC Shanghai Aircraft Manufacturing Co., Ltd., Shanghai 201324, China
Funding
National Key R&D Program of China
References (34)
Show source-provided references
1. Configuration and implementation of a daily artificial neural network-based forecasting system using real supermarket data
2. Kim, M., Lee, J., Lee, C., and Jeong, J. (2022). Framework of 2d kde and lstm-based forecasting for cost-effective inventory management in smart manufacturing. Appl. Sci., 12.10.3390/app12052380 ↗
3. A grey-layered ANP based decision support model for analyzing strategies of resilience in electronic supply chains10.1016/j.engappai.2019.103338 ↗
4. A novel learning based approach for a new integrated location-routing and scheduling problem within cross-docking considering direct shipment10.1016/j.asoc.2015.04.062 ↗
5. Cantini, A., Peron, M., De Carlo, F., and Sgarbossa, F. (2022). A decision support system for configuring spare parts supply chains considering different manufacturing technologies. Int. J. Prod. Res., 1–21.10.1080/00207543.2022.2041757 ↗
6. Taboada, H., Davizón, Y.A., Espíritu, J.F., and Sánchez-Leal, J. (2022). Mathematical Modeling and Optimal Control for a Class of Dynamic Supply Chain: A Systems Theory Approach. Appl. Sci., 12.10.3390/app12115347 ↗
7. Afsar, H.M., Ben-Ammar, O., Dolgui, A., and Hnaien, F. (2020). Supplier replacement model in a one-level assembly system under lead-time uncertainty. Appl. Sci., 10.10.20944/preprints202003.0322.v1 ↗
8. A predictive integrated genetic-based model for supplier evaluation and selection10.1007/s40815-017-0324-z ↗
9. Reward shaping to improve the performance of deep reinforcement learning in perishable inventory management10.1016/j.ejor.2021.10.045 ↗
10. A survey on federated learning10.1016/j.knosys.2021.106775 ↗
11. Inventory management in supply chains: A reinforcement learning approach10.1016/S0925-5273(00)00156-0 ↗
12. Hybrid algorithm based on reinforcement learning for smart inventory management10.1007/s10845-022-01982-5 ↗
13. Reinforcement learning approach for efficient inventory policy in multi-echelon supply chain under various assumptions and constraints10.20473/jisebi.7.2.138-148 ↗
14. Case-based reinforcement learning for dynamic inventory control in a multi-agent supply-chain system10.1016/j.eswa.2008.07.036 ↗
15. Oroojlooy, A. (2019). Applications of Machine Learning in Supply Chains. [Ph.D. Thesis, Lehigh University].
16. Kemmer, L., von Kleist, H., de Rochebouët, D., Tziortziotis, N., and Read, J. (2018, January 1–3). Reinforcement learning for supply chain optimization. Proceedings of the European Workshop on Reinforcement Learning, Lille, France.
17. Hutse, V., Verleysen, A., and Wyffels, F. (2019). Reinforcement Learning for Inventory Optimisation in Multi-Echelon Supply Chains, Master in Business Engineering—Ghent University.
18. Alves, J.C., and Mateus, G.R. (2020, January 28–30). Deep reinforcement learning and optimization approach for multi-echelon supply chain with uncertain demands. Proceedings of the International Conference on Computational Logistics, Enschede, The Netherlands.10.1007/978-3-030-59747-4_38 ↗
19. Alves, J.C., Silva, D.M.d., and Mateus, G.R. (2021, January 21–23). Applying and comparing policy gradient methods to multi-echelon supply chains with uncertain demands and lead times. Proceedings of the International Conference on Artificial Intelligence and Soft Computing, Virtual Event.10.1007/978-3-030-87897-9_21 ↗
20. Dynamic inventory replenishment strategy for aerospace manufacturing supply chain: Combining reinforcement learning and multi-agent simulation10.1080/00207543.2021.2020927 ↗
21. Reinforcement learning approaches for specifying ordering policies of perishable inventory systems10.1016/j.eswa.2017.08.046 ↗
22. Abu Zwaida, T., Pham, C., and Beauregard, Y. (2021). Optimization of inventory management to prevent drug shortages in the hospital supply chain. Appl. Sci., 11.10.3390/app11062726 ↗
23. Designing of an intelligent self-adaptive model for supply chain ordering management system10.1016/j.engappai.2014.09.004 ↗
24. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv.
25. Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 15–15). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Proceedings of the International Conference on Machine Learning, PMLR, Stockholm, Sweden.
26. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv.
27. Barat, S., Khadilkar, H., Meisheri, H., Kulkarni, V., Baniwal, V., Kumar, P., and Gajrani, M. (2019, January 13–17). Actor based simulation for closed loop control of supply chain using reinforcement learning. Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems, Montreal, QC, Canada.
28. Sultana, N.N., Meisheri, H., Baniwal, V., Nath, S., Ravindran, B., and Khadilkar, H. (2020). Reinforcement learning for multi-product multi-node inventory management in supply chains. arXiv.
29. Inventory management of new products in retailers using model-based deep reinforcement learning10.1016/j.eswa.2023.120256 ↗
30. Jullien, S., Ariannezhad, M., Groth, P., and de Rijke, M. (Trans. Mach. Learn. Res., 2023). A simulation environment and reinforcement learning method for waste reduction, Trans. Mach. Learn. Res., in press.
31. Yu, C., Velu, A., Vinitsky, E., Wang, Y., Bayen, A., and Wu, Y. (2021). The surprising effectiveness of ppo in cooperative, multi-agent games. arXiv.
32. Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., and Whiteson, S. (2018, January 2–7). Counterfactual multi-agent policy gradients. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11794 ↗
33. Son, K., Kim, D., Kang, W.J., Hostallero, D.E., and Yi, Y. (2019, January 9–15). Qtran: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA.
34. Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. (2015). High-dimensional continuous control using generalized advantage estimation. arXiv.