Deep Reinforcement Learning for Selection of Dispatch Rules for Scheduling of Production Systems
Source abstract
Abstract
Production scheduling is a critical task in the management of manufacturing systems. It is difficult to derive an optimal schedule due to the problem complexity. Computationally expensive and time-consuming solutions have created major issues for companies trying to respect their customers’ demands. Simple dispatching rules have typically been applied in manufacturing practice and serve as a good scheduling option, especially for small and midsize enterprises (SMEs). However, in recent years, the progress in smart systems enabled by artificial intelligence (AI) and machine learning (ML) solutions has revolutionized the scheduling approach. Under different production circumstances, one dispatch rule may perform better than others, and expert knowledge is required to determine which rule to choose. The objective of this work is to design and implement a framework for the modeling and deployment of a deep reinforcement learning (DRL) agent to support short-term production scheduling. The DRL agent selects a dispatching rule to assign jobs to manufacturing resources. The model is trained, tested and evaluated using a discrete event simulation (DES) model that simulates a pilot case from the bicycle production industry. The DRL agent can learn the best dispatching policy, resulting in schedules with the best possible production makespan.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Scheduling and Optimization Algorithms · Advanced Control Systems Optimization · Advanced Manufacturing and Logistics Optimization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Laboratory for Manufacturing Systems & Automation (LMS), Department of Mechanical Engineering & Aeronautics, University of Patras, Rio, 26504 Patras, Greece
Funding
European Union’s Horizon 2020 research and innovation program
References (62)
Show source-provided references
1. Artificial Intelligence in Manufacturing Systems10.1007/978-3-031-21828-6_4 ↗
2. Mourtzis, D. (2022). Advances in Adaptive Scheduling in Industry 4.0. Front. Manuf. Technol., 2.10.3389/fmtec.2022.937889 ↗
3. Cioffi, R., Travaglioni, M., Piscitelli, G., Petrillo, A., and Parmentola, A. (2020). Smart Manufacturing Systems and Applied Industrial Technologies for a Sustainable Industry: A Systematic Literature Review. Appl. Sci., 10.10.3390/app10082897 ↗
4. Industry 4.0 technologies assessment: A sustainability perspective10.1016/j.ijpe.2020.107776 ↗
5. Chryssolouris, G. (2013). Manufacturing Systems: Theory and Practice, Springer.
6. Dynamic scheduling of manufacturing job shops using genetic algorithms10.1023/A:1011253011638 ↗
7. A cloud-based cyber-physical system for adaptive shop-floor scheduling and condition-based maintenance10.1016/j.jmsy.2018.05.008 ↗
8. Integrated production scheduling and process control: A systematic review10.1016/j.compchemeng.2014.09.002 ↗
9. Real-time production scheduling in the Industry-4.0 context: Addressing uncertainties in job arrivals and machine breakdowns10.1016/j.cor.2020.105031 ↗
10. Digital twin-driven supervised machine learning for the development of artificial intelligence applications in manufacturing10.1080/0951192X.2020.1747642 ↗
11. Framinan, J.M., Leisten, R., and García, R.R. (2014). Manufacturing Scheduling Systems: An Integrated View on Models, Methods and Tools, Springer.10.1007/978-1-4471-6272-8 ↗
12. Dynamic scheduling selection of dispatching rules for manufacturing system10.1080/002075497195137 ↗
13. Dispatching rules for dynamic scheduling of one-of-a-kind production10.1080/09511920500407541 ↗
14. Review of job shop scheduling research and its new perspectives under Industry 4.010.1007/s10845-017-1350-2 ↗
15. Resource planning for the installation of industrial product service systems
16. Refinery short-term scheduling with tank farm, inventory and distillation management: An integrated simulation-based approach10.1016/j.ejor.2004.03.046 ↗
17. Real-time scheduling simulation optimisation of job shop in a production-logistics collaborative environment10.1080/00207543.2021.2023777 ↗
18. Unified Multi-Objective Genetic Algorithm for Energy Efficient Job Shop Scheduling10.1109/ACCESS.2021.3070981 ↗
19. A heuristic for adaptive production scheduling and control in flow shop production10.1080/00207540903575385 ↗
20. A two-layer dynamic scheduling method for minimising the earliness and tardiness of a re-entrant production line10.1080/00207543.2010.543171 ↗
21. Dynamic scheduling method for integrated process planning and scheduling problem with machine fault10.1016/j.rcim.2022.102334 ↗
22. Intelligent scheduling of discrete automated production line via deep reinforcement learning10.1080/00207543.2020.1717008 ↗
23. An integrated scheduling approach considering dispatching strategy and conflict-free route of AMRs in flexible job shop10.1007/s00170-022-10619-z ↗
24. Digital twin-driven service model and optimal allocation of manufacturing resources in shared manufacturing10.1016/j.jmsy.2021.02.008 ↗
25. A heuristic model for dynamic flexible job shop scheduling problem considering variable processing times10.1080/00207543.2018.1524165 ↗
26. New meta-heuristic for dynamic scheduling in permutation flowshop with new order arrival10.1007/s00170-018-2171-y ↗
27. Hybrid task scheduling strategy for cloud computing by modified particle swarm optimization and fuzzy theory10.1016/j.cie.2019.03.006 ↗
28. Makespan-Driven Workflow Scheduling in Clouds Using Immune-Based PSO Algorithm10.1109/ACCESS.2020.2972963 ↗
29. A Knowledge-Based Cooperative Algorithm for Energy-Efficient Scheduling of Distributed Flow-Shop10.1109/TSMC.2017.2788879 ↗
30. Learning to schedule job-shop problems: Representation and policy learning using graph neural network and reinforcement learning10.1080/00207543.2020.1870013 ↗
31. Optimization and Simulation of Remanufacturing Production Scheduling under Uncertainties10.2507/IJSIMM17(4)CO20 ↗
32. Shahzad, A., and Mebarki, N. (2016). Learning Dispatching Rules for Scheduling: A Synergistic View Comprising Decision Trees, Tabu Search and Simulation. Computers, 5.10.3390/computers5010003 ↗
33. A hybrid many-objective evolutionary algorithm for flexible job-shop scheduling problem with transportation and setup times10.1016/j.cor.2021.105263 ↗
34. A Review of Dynamic Job Shop Scheduling Techniques10.1016/j.promfg.2019.02.006 ↗
35. Dynamic scheduling of manufacturing systems using machine learning: An updated review10.1017/S0890060413000516 ↗
36. A toolbox of agents for scheduling the paint shop in bicycle industry10.1016/j.procir.2022.05.124 ↗
37. Deep Reinforcement Learning for autonomous pre-failure tool life improvement10.1007/s00170-022-09700-4 ↗
38. Deep reinforcement learning applied to an assembly sequence planning problem with user preferences10.1007/s00170-022-09877-8 ↗
39. Torres, A.d.R., Andreiana, D.S., Roldán, O., Bustos, A.H., and Galicia, L.E.A. (2022). A Review of Deep Reinforcement Learning Approaches for Smart Manufacturing in Industry 4.0 and 5.0 Framework. Appl. Sci., 12.10.3390/app122312377 ↗
40. Panzer, M., Bender, B., and Gronau, N. (2021, January 10–11). Deep Reinforcement Learning In Production Planning And Control: A Systematic Literature Review. Proceedings of the Conference on Production Systems and Logistics, Online.
41. Cunha, B., Madureira, A., Fonseca, B., and Matos, J. (2021). Intelligent Scheduling with Reinforcement Learning. Appl. Sci., 11.10.3390/app11083710 ↗
42. Training a neural network to select dispatching rules in real time10.1016/j.cie.2009.03.008 ↗
43. Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press.
44. Hubbs, C., Wassick, J.M., Hubbs, C.D., Kelloway, A., Sahinidis, N.V., and Grossmann, I.E. (2020). An Industrial Application of Deep Reinforcement Learning for Chemical Production Scheduling. Machine Learning for Engineering Modeling, Simulation, and Design, Researchgate.net.10.1016/j.compchemeng.2020.106982 ↗
45. Reinforcement Learning with Composite Rewards for Production Scheduling in a Smart Factory10.1109/ACCESS.2020.3046784 ↗
46. Smart scheduling of dynamic job shop based on discrete event simulation and deep reinforcement learning10.1007/s10845-023-02161-w ↗
47. A Deep Reinforcement Learning Based Scheduling Policy for Reconfigurable Manufacturing Systems10.1016/j.procir.2021.09.089 ↗
48. Dynamic scheduling in a job-shop production system with reinforcement learning10.1016/j.procir.2020.05.210 ↗
49. Deep reinforcement learning-based dynamic scheduling in smart manufacturing
50. Learning policies for single machine job dispatching10.1016/j.rcim.2004.07.003 ↗
51. Petri-net-based dynamic scheduling of flexible manufacturing system via deep reinforcement learning with graph convolutional network10.1016/j.jmsy.2020.02.004 ↗
52. Zhang, C., Song, W., Cao, Z., Zhang, J., Tan, P.S., and Xu, C. (2020, January 6–12). Learning to dispatch for job shop scheduling via deep reinforcement learning. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, BC, Canada.
53. Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning10.1016/j.asoc.2020.106208 ↗
54. Dynamic production scheduling towards self-organizing mass personalization: A multi-agent dueling deep reinforcement learning approach10.1016/j.jmsy.2023.03.003 ↗
55. An adaptive multi-objective multi-task scheduling method by hierarchical deep reinforcement learning10.1016/j.asoc.2024.111342 ↗
56. Zeng, Y., Liao, Z., Dai, Y., Wang, R., Li, X., and Yuan, B. (2022). Hybrid intelligence for dynamic job-shop scheduling with deep reinforcement learning and attention mechanism. arXiv.
57. Smart Manufacturing Scheduling With Edge Computing Using Multiclass Deep Q Network10.1109/TII.2019.2908210 ↗
58. Optimization of global production scheduling with deep reinforcement learning10.1016/j.procir.2018.03.212 ↗
59. Q-learning10.1007/BF00992698 ↗
60. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
61. The role of simulation in digital manufacturing: Applications and outlook10.1080/0951192X.2013.800234 ↗
62. Simulation in the design and operation of manufacturing systems: State of the art and new trends10.1080/00207543.2019.1636321 ↗