journal article·2025-06-11Open access

Multi-Scenario Robust Distributed Permutation Flow Shop Scheduling Based on DDQN

Shilong Guo, Ming Chen

Applied Sciences· Volume 15· Issue 12

Abstract

In order to address the Distributed Displacement Flow Shop Scheduling Problem (DPFSP) with uncertain processing times in real production environments, Plant Simulation is employed to construct a simulation model for the MSRDPFSP. The model conducts quantitative analyses of workshop layout, assembly line design, worker status, operating status of robotic arms and AGV vehicles, and production system failure rates. A hybrid NEH-DDQN algorithm is integrated into the simulation model via a COM interface and DLL, where the NEH algorithm ensures the model maintains optimal performance during the early training phase. Four scheduling strategies are designed for workpiece allocation across different workshops. A deep neural network replaces the traditional Q-table for greedy selection among these four scheduling strategies, using each workshop’s completion time as a simplified state variable. This approach reduces algorithm training complexity by abstracting away intricate workpiece allocation details. Experimental comparisons show that for the data of 500 workpieces, the NEH algorithm in 3 s demonstrates equivalent quality to that produced by the GA algorithm in 300 s. After 2000 iterations, the DDQN algorithm achieves a 15% reduction in makespan with only a 2.5% increase in computational time compared to random search, this joint simulation system offers an efficient and stable solution for the modeling and optimization of the MSRDPFSP issue.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Scheduling and Optimization Algorithms · Advanced Manufacturing and Logistics Optimization · Digital Transformation in Industry

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Key Laboratory of Fisheries Information, Ministry of Agriculture and Rural Affairs, College of Information Technology, Shanghai Ocean University, Hucheng Ring Road 999, Shanghai 201306, China

References (46)

Show source-provided references
1. Industry 4.0, digitization, and opportunities for sustainability10.1016/j.jclepro.2019.119869 ↗
2. Zimmermann, A., Schmidt, R., Sandkuhl, K., Wißotzki, M., Jugel, D., and Möhring, M. (2015, January 21–25). Digital enterprise architecture-transformation for the internet of things. Proceedings of the IEEE 19th International Enterprise Distributed Object Computing Workshop, Adelaide, Australia.10.1109/EDOCW.2015.16 ↗
3. The distributed permutation flowshop scheduling problem10.1016/j.cor.2009.06.019 ↗
4. The hybrid flow shop scheduling problem10.1016/j.ejor.2009.09.024 ↗
5. Asimulation-based study of dispatching rules in a dynamic job shop scheduling problem with batch release and extended technical precedence constraints10.1016/j.ejor.2016.07.030 ↗
6. A competitive memetic algorithm for multi-objective distributed permutation flow shop scheduling problem10.1016/j.swevo.2016.06.002 ↗
7. A genetic algorithm embedded with a concise chromosome representation for distributed and flexible job-shop scheduling problems10.1007/s10845-015-1083-z ↗
8. Iterated Greedy methods for the distributed permutation flowshop scheduling problem10.1016/j.omega.2018.03.004 ↗
9. Gogos, C. (2023). Solving the Distributed Permutation Flow-Shop Scheduling Problem Using Constrained Programming. Appl. Sci., 13.10.3390/app132312562 ↗
10. Self-adaptive population-based iterated greedy algorithm for distributed permutation flowshop scheduling problem with part of jobs subject to a common deadline constraint10.1016/j.eswa.2024.123278 ↗
11. Q-Learning-Based Priority Dispatching Rule Preference Model for Non-Permutation Flow Shop10.1142/S0219686724500252 ↗
12. Wang, Y., Qian, B., Hu, R., Yang, Y., and Chen, W. (2023). Deep Reinforcement Learning for Solving Distributed Permutation Flow Shop Scheduling Problem. Advanced Intelligent Computing Technology and Applications, Proceedings of the 19th International Conference, ICIC 2023, Zhengzhou, China, 10–13 August 2023, Springer.10.1007/978-981-99-4755-3_29 ↗
13. A distributed permutation flowshop scheduling problem with the customer order constraint10.1016/j.knosys.2019.104894 ↗
14. A hybrid local-search algorithm for robust job-shop scheduling under scenarios10.1016/j.asoc.2017.10.020 ↗
15. Zhang, J., and Cai, J. (2023). A dual-population genetic algorithm with Q-learning for multi-objective distributed hybrid flow shop scheduling problem. Symmetry, 15.10.3390/sym15040836 ↗
16. Zhou, T., Luo, L., Ji, S., and He, Y. (2023). A reinforcement learning approach to robust scheduling of permutation flow shop. Biomimetics, 8.10.3390/biomimetics8060478 ↗
17. On reliability of reinforcement learning based production scheduling systems: A comparative survey10.1007/s10845-022-01915-2 ↗
18. Robust job-shop scheduling under deterministic and stochastic unavailability constraints due to preventive and corrective maintenance10.1016/j.cie.2022.108130 ↗
19. Scheduling of stochastic distributed hybrid flow-shop by hybrid estimation of distribution algorithm and proximal policy optimization10.1016/j.eswa.2025.126523 ↗
20. Dynamic scheduling mechanism for intelligent workshop with deep reinforcement learning method based on multi-agent system architecture10.1016/j.cie.2024.110155 ↗
21. Increasing the production capacity of business processes using plant simulation10.2507/IJSIMM23-1-669 ↗
22. Fedorko, G., Molnár, V., Strohmandl, J., Horváthová, P., Strnad, D., and Cech, V. (2022). Research on using the tecnomatix plant simulation for simulation and visualization of traffic processes at the traffic node. Appl. Sci., 12.10.3390/app122312131 ↗
23. Developing simulation approaches: A simple case of emulation for logic validation using tecnomatix plant simulation10.1088/1742-6596/2212/1/012011 ↗
24. Zhao, W.B., Hu, J.H., and Tang, Z.Q. (2024). Virtual Simulation-Based Optimization for Assembly Flow Shop Scheduling Using Migratory Bird Algorithm. Biomimetics, 9.10.3390/biomimetics9090571 ↗
25. Milk run testing through Tecnomatix plant simulation software10.2507/IJSIMM21-1-593 ↗
26. Modelling an assembly line using tecnomatix plant simulation software
27. Wesch, J.O. (2022). An Implementation Strategy for Tecnomatix Plant Simulation Software. [Master’s Thesis, North-West University].
28. Flow shop scheduling problems with assembly operations: A review and new trends10.1080/00207543.2018.1550269 ↗
29. Application of tecnomatix plant simulation for modeling production and logistics processes10.3846/bme.2016.316 ↗
30. Bangsow, S. (2020). Tecnomatix Plant Simulation, Springer International Publishing.10.1007/978-3-030-41544-0 ↗
31. Identification of production bottlenecks with the use of plant simulation software
32. An NEH-based heuristic algorithm for distributed permutation flowshop scheduling problems
33. del Real Torres, A., Andreiana, D.S., Ojeda Roldán, Á., Bustos, A.H., and Galicia, L.E.A. (2022). A review of deep reinforcement learning approaches for smart manufacturing in industry 4.0 and 5.0 framework. Appl. Sci., 12.10.3390/app122312377 ↗
34. Research on adaptive job shop scheduling problems based on dueling double DQN10.1109/ACCESS.2020.3029868 ↗
35. Intelligent manufacturing in the context of industry 4.0: A review10.1016/J.ENG.2017.05.015 ↗
36. Optimization of production system in plant simulation10.1177/00375497211038908 ↗
37. Distributed permutation flow shop scheduling problem with worker flexibility: Review, trends and model proposition10.1016/j.eswa.2023.121947 ↗
38. A distributed flexible job shop scheduling problem considering worker arrangement using an improved memetic algorithm10.1016/j.eswa.2022.117984 ↗
39. Deep reinforcement learning based AGVs real-time scheduling with mixed rule for flexible shop floor in industry 4.010.1016/j.cie.2020.106749 ↗
40. Research on integrated scheduling of AGV and machine in flexible job shop
41. Scheduling in job shops with machine breakdowns: An experimental study10.1016/S0360-8352(99)00006-6 ↗
42. NEH-based heuristics for the permutation flowshop scheduling problem to minimise total tardiness10.1016/j.cor.2015.02.002 ↗
43. Smart packet transmission scheduling in cognitive IoT systems: DDQN based approach10.1109/ACCESS.2022.3168549 ↗
44. Li, J., Wang, P., Li, Q., He, Y., Sun, L., Sun, Y., Zhao, P., and Jia, L. (2023). Research on Application Automation Operations Based on Win32com. Advances in Artificial Intelligence, Big Data and Algorithms, IOS Press.10.3233/FAIA230799 ↗
45. Towers, M., Kwiatkowski, A., Terry, J., Balis, J.U., De Cola, G., Deleu, T., Goulão, M., Kallinteris, A., Krimmel, M., and Arjun, K.G. (2024). Gymnasium: A standard interface for reinforcement learning environments. arXiv.
46. Schäfer, G., Schirl, M., Rehrl, J., Huber, S., and Hirlaender, S. (2024, January 3–6). Python-Based Reinforcement Learning on Simulink Models. Proceedings of the International Conference on Soft Methods in Probability and Statistics, Salzburg, Austria.10.1007/978-3-031-65993-5_55 ↗