Deep Reinforcement Learning Approach for Material Scheduling Considering High-Dimensional Environment of Hybrid Flow-Shop Problem
Source abstract
Abstract
Manufacturing sites encounter various scheduling problems, which must be dealt with to efficiently manufacture products and reduce costs. With the development of smart factory technology, many elements at manufacturing sites have become unmanned and more complex. Moreover, owing to the mixing of several processes in one production line, the need for efficient scheduling of materials has emerged. The aim of this study is to solve the material scheduling problem of many machines in a hybrid flow-shop environment using deep reinforcement learning. Most previous work has ignored some conditions, which were critical for solving practical problems. Such critical conditions make the scheduling more complex and difficult to solve. They expand the size of the state and large action space and make learning in an environment with many machines problematic. In this study, a reinforcement learning approach was developed considering practical factors such as the processing time and material transfer to solve realistic manufacturing scheduling problems. Additionally, a method to simplify the high-dimensional environmental space at manufacturing sites for efficient learning was established to solve the problem of learning in a high-dimensional space. Through experiments, we showed that our approach could optimally schedule material scheduling in multi-process lines, which contributes to realistic manufacturing intelligence.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Scheduling and Optimization Algorithms · Advanced Manufacturing and Logistics Optimization · Assembly Line Balancing Optimization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Digital Media and Communications Engineering, Sungkyunkwan University, Suwon 16419, Korea
Department of Artificial Intelligence, Sungkyunkwan University, Suwon 16419, Korea
Funding
Korea government (MSIT)
References (37)
Show source-provided references
1. Solving the hybrid flow shop scheduling problem using heuristic algorithms
2. Parallel tabu search algorithm for the hybrid flow shop problem10.1016/j.cie.2013.04.007 ↗
3. A Genetic Algorithm for Three-Stage Hybrid Flow Shop Scheduling Problem with Dedicated Machines10.18280/jesa.530306 ↗
4. Umam, M.S., Mustafid, M., and Suryono, S. (J. King Saud Univ. Comput. Inf. Sci., 2021). A hybrid genetic algorithm and tabu search for minimizing makespan in flow shop scheduling problem, J. King Saud Univ. Comput. Inf. Sci., in press.10.1016/j.jksuci.2021.08.025 ↗
5. Scheduling in aerospace composite manufacturing systems: A two-stage hybrid flow shop problem10.1007/s00170-017-1429-0 ↗
6. Tobias, R., Abdulrahman, N., Fabian, B., and Sebastian, L. (2020, January 7–10). Evolving Neural Networks to Solve a Two-Stage Hybrid Flow Shop Scheduling Problem with Family Setup Times. Proceedings of the 53rd Hawaii International Conference on System Sciences, Maui, HI, USA.
7. Real-Time Hybrid Flow Shop Scheduling Approach in Smart Manufacturing Environment10.23919/CSMS.2021.0024 ↗
8. A research survey: Review of flexible job shop scheduling techniques10.1111/itor.12199 ↗
9. Yang, S., Xu, Z., and Wang, J. (2021). Intelligent Decision-Making of Scheduling for Dynamic Permutation Flowshop via Deep Reinforcement Learning. Sensors, 21.10.3390/s21031019 ↗
10. Review and classification of hybrid shop scheduling10.1007/s11740-018-0832-1 ↗
11. Six decades of flowshop scheduling research
12. An Improvement of Reinforcement Learning Approach for Permutation of Flow-Shop Scheduling Problems
13. Yang, S., and Xu, Z. (2021, January 12–14). Intelligent Scheduling for Permutation Flow Shop with Dynamic Job Arrival via Deep Reinforcement Learning. Proceedings of the 2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, China.10.1109/IAEAC50856.2021.9390893 ↗
14. Han, W., Guo, F., and Su, X. (2019). A reinforcement learning method for a hybrid flow-shop scheduling problem. Algorithms, 12.10.3390/a12110222 ↗
15. Solving flow-shop scheduling problem with a reinforcement learning algorithm that generalizes the value function with neural network10.1016/j.aej.2021.01.030 ↗
16. Review of job shop scheduling research and its new perspectives under Industry 4.010.1007/s10845-017-1350-2 ↗
17. Jones, A., Rabelo, L.C., and Sharawi, A.T. (1998). Survey of Job Shop Scheduling Techniques, NISTIR, National Institute of Standards and Technology.10.1002/047134608X.W3352 ↗
18. Review on job-shop and flow-shop scheduling using10.3329/jme.v41i2.7508 ↗
19. Hybrid flow shop scheduling: A survey10.1016/S0360-8352(99)00023-6 ↗
20. The hybrid flow shop scheduling problem10.1016/j.ejor.2009.09.024 ↗
21. Lourenco, H. (1993). A Computational Study of the Job-Shop and the Flow-Shop Scheduling Problems, Cornell University.
22. Kochhar, A.K. (April, January 31st). Flow-Shop Scheduling: Effect of Various Priority Rules on Minimizing Multiple Criteria. Proceedings of the Thirtieth International MATADOR Conference, Manchester, UK.
23. Brucker, P. (2007). Scheduling Algorithms, Springer. [5th ed.].
24. Brucker, P., Lenstra, J.K., and Kan, A.H.G.R. (1975). Complexity of Machine Scheduling Problems, Stichting Mathematisch Centrum.
25. A Survey of Scheduling Rules10.1287/opre.25.1.45 ↗
26. A Reinforcement Learning Method to Scheduling Problem of Steel Production Process10.1088/1742-6596/1486/7/072035 ↗
27. Flow shop scheduling with reinforcement learning10.1142/S0217595913500140 ↗
28. Q-learning algorithm performance for m-machine, n-jobs flow shop scheduling problems to minimize makespan
29. Pipple, R.R. (2019). The Application of Deep Reinforcement Learning to the Hybrid Flow-Shop Scheduling Problem. [Ph.D. Thesis, Tilburg University].
30. Lamprecht, R., Wurst, F., and Huber, M.F. (2021, January 21–23). Reinforcement Learning based Condition-oriented Maintenance Scheduling for Flow Line Systems. Proceedings of the 2021 IEEE 19th International Conference on Industrial Informatics (INDIN), Palma de Mallorca, Spain.10.1109/INDIN45523.2021.9557373 ↗
31. Tassel, P., Gebser, M., and Schekotihin, K. (2021). A Reinforcement Learning Environment for Job-Shop Scheduling. arXiv.
32. Adaptive Reactive Job-Shop Scheduling with Reinforcement Learning Agents
33. Reinforcement learning approach to scheduling of precast concrete production10.1016/j.jclepro.2022.130419 ↗
34. Distributed policy search reinforcement learning for job-shop scheduling tasks10.1080/00207543.2011.571443 ↗
35. Van Otterlo, M., and Wiering, M. (2012). Reinforcement learning and markov decision processes. Reinforcement Learning, Springer.10.1007/978-3-642-27645-3_1 ↗
36. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
37. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv.