Learning-Based Optimisation for Integrated Problems in Intermodal Freight Transport: Preliminaries, Strategies, and State of the Art
Source abstract
Abstract
Intermodal freight transport (IFT) requires a large number of optimisation measures to ensure its attractiveness. This involves numerous control decisions on different time scales, making integrated optimisation with traditional methods almost unfeasible. Recently, a new trend in optimisation science has emerged: the application of Deep Learning (DL) to combinatorial problems. Neural combinatorial optimisation (NCO) enables real-time decision-making under uncertainties by considering rich context information—a crucial factor for seamless synchronisation, optimisation, and, consequently, for the competitiveness of IFT. The objective of this study is twofold. First, we systematically analyse and identify the key actors, operations, and optimisation problems in IFT and categorise them into six major classes. Second, we collect and structure the key methodological components of the NCO framework, including DL models, training algorithms, design strategies, and review the current State of the Art with a focus on NCO and hybrid DL models. Through this synthesis, we integrate the latest research efforts from three closely related fields: optimisation, transport planning, and NCO. Finally, we critically discuss and outline methodological design patterns and derive potential opportunities and obstacles for learning-based frameworks for integrated optimisation problems. Together, these efforts aim to enable a better integration of advanced DL techniques into transport logistics. We hope that this will help researchers and practitioners in related fields to expand their intuition and foster the development of intelligent decision-making systems and algorithms for tomorrow’s transport systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Maritime Ports and Logistics · Transportation Planning and Optimization · Vehicle Routing Optimization Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
German Aerospace Center (DLR), Institute of Transport Research, Rudower Chaussee 7, 12489 Berlin, Germany
Funding
German Federal Ministry for Digital and Transport (BMDV)
References (100+)
Show source-provided references
1. Analysis of mode choice variables in short-distance intermodal freight transport using an agent-based model10.1016/j.tra.2014.01.002 ↗
2. Machine learning for international freight transportation management: A comprehensive review
3. Bešinović, N., and Goverde, R.M. (2018). Capacity assessment in railway networks. Handbook of Optimization in the Railway Industry, Springer.10.1007/978-3-319-72153-8_2 ↗
4. Multimodal freight transportation planning: A literature review10.1016/j.ejor.2013.06.055 ↗
5. Tang, Y., Agrawal, S., and Faenza, Y. (2020, January 13–18). Reinforcement learning for integer programming: Learning to cut. Proceedings of the International Conference on Machine Learning, Virtual.
6. Nair, V., Bartunov, S., Gimeno, F., Von Glehn, I., Lichocki, P., and Zwols, Y. (2020). Solving mixed integer programs using neural networks. arXiv.
7. Kotary, J., Fioretto, F., Van Hentenryck, P., and Wilder, B. (2021). End-to-end constrained optimization learning: A survey. arXiv.10.24963/ijcai.2021/610 ↗
8. Reinforcement learning for combinatorial optimization: A survey10.1016/j.cor.2021.105400 ↗
9. Machine learning at the service of meta-heuristics for solving combinatorial optimization problems: A state-of-the-art10.1016/j.ejor.2021.04.032 ↗
10. Vaswani, A., Shazeer, N.M., Parmar, N., Uszkoreit, J., and Polosukhin, I. (2017). Attention is All you Need. Adv. Neural Inf. Process. Syst., 30, Available online: https://dl.acm.org/doi/10.5555/3295222.3295349.
11. Intermodal transport in freight distribution: A literature review10.1080/01441647.2017.1297868 ↗
12. Rail-Roadmap (2023, June 01). Level Playing Field in the Transport Sector. Available online: https://www.railroadmap2030.be/wp-content/uploads/2021/09/BRFF-Level-playing-field-in-the-transport-sector.pdf.
13. Agasucci, V., Grani, G., and Lamorgese, L. (2020). Solving the single-track train scheduling problem via Deep Reinforcement Learning. arXiv.
14. Dynamic approach to solve the daily drayage problem with transit time uncertainty10.1016/j.compind.2012.11.006 ↗
15. Bektas, T., and Crainic, T. (2007). A Brief Overview of Intermodal Transportation, Cirrelt.10.1016/S0927-0507(06)14008-6 ↗
16. Synchromodal logistics: An overview of critical success factors, enabling technologies, and open research issues10.1016/j.tre.2019.07.009 ↗
17. Transit network design and scheduling: A global review10.1016/j.tra.2008.03.011 ↗
18. Selecting a location for establishing a rail-road intermodal terminal by using a hybrid SWOT/MCDM model10.1088/1757-899X/618/1/012060 ↗
19. Using heuristic MCMC method for terminal location planning in intermodal transportation10.1504/IJOR.2018.093506 ↗
20. Centralized and decentralized train scheduling for intermodal operations10.1080/07408170008967432 ↗
21. Multimodal schedule design for synchromodal freight transport systems
22. Weik, N., Bohlin, M., and Nießen, N. (2024, September 20). Long-Term Capacity Planning of Railway Infrastructure: A Stochastic Approach Capturing Infrastructure Unavailability. RWTH Aachen University. PhD Thesis No.RWTH-2020-06771. Lehrstuhl für Schienenbahnwesen und Verkehrswirtschaft und Verkehrswissenschaftliches Institut. Available online: https://publications.rwth-aachen.de/record/793271/files/793271.pdf.
23. Li, Z., Huang, P., Wen, C., and Rodrigues, F. (2023). Railway Network Delay Evolution: A Heterogeneous Graph Neural Network Approach. arXiv.10.2139/ssrn.4183321 ↗
24. Integrating vehicle routing into intermodal service network design with stochastic transit times10.1016/j.ejtl.2021.100046 ↗
25. Intermodal service network design with stochastic demand and short-term schedule modifications10.1016/j.cie.2021.107514 ↗
26. The evolution of intermodal transport research and its development issues10.3846/16484142.2007.9638145 ↗
27. Baykasoğlu, A., Subulan, K., Serdar Taşan, A., and Ülker, Ö. (2018). Development of a Web-Based Decision Support System for Strategic and Tactical Sustainable Fleet Management Problems in Intermodal Transportation Networks. Lean and Green Supply Chain Management, Springer.10.1007/978-3-319-97511-5_7 ↗
28. Facility location dynamics: An overview of classifications and applications10.1016/j.cie.2011.09.018 ↗
29. Approximation algorithms for network design: A survey
30. A survey of optimization models for train routing and scheduling10.1287/trsc.32.4.380 ↗
31. A survey of transportation problems
32. Feeney, G.J. (1959). The Distribution of Empty Freight Cars, Columbia University.
33. A model for fleet sizing and vehicle allocation10.1287/trsc.25.1.19 ↗
34. A review of fleet planning problems in single and multimodal transportation systems
35. Zhang, H., Ge, H., Yang, J., and Tong, Y. (2021). Review of vehicle routing problems: Models, classification and solving algorithms. Archives of Computational Methods in Engineering, Springer.10.1007/s11831-021-09574-x ↗
36. Golden, B., Wang, X., and Wasil, E. (2023). The Evolution of the Vehicle Routing Problem—A Survey of VRP Research and Practice from 2005 to 2022, Springer.10.1007/978-3-031-18716-2_1 ↗
37. Order batching and sequencing for the minimization of the total tardiness in picker-to-part warehouses10.1007/s10696-012-9164-1 ↗
38. Hu, Q., Corman, F., and Lodewijks, G. (2015, January 23–25). A review of intermodal rail freight bundling operations. Proceedings of the Computational Logistics: 6th International Conference, ICCL 2015, Delft, The Netherlands.10.1007/978-3-319-24264-4_31 ↗
39. A data-driven two-stage prediction model for train primary-delay recovery time10.1142/S0218194020400124 ↗
40. Demand forecasting in supply chains: A review of aggregation and hierarchical approaches10.1080/00207543.2021.2005268 ↗
41. Shah, N.H., and Mittal, M. (2020). Optimization and Inventory Management, Springer.10.1007/978-981-13-9698-4 ↗
42. Perez, H.D., Hubbs, C.D., Li, C., and Grossmann, I.E. (2021). Algorithmic approaches to inventory management optimization. Processes, 9.10.3390/pr9010102 ↗
43. Bello, I., Pham, H., Le, Q.V., Norouzi, M., and Bengio, S. (2017, January 24–26). Neural combinatorial optimization with reinforcement learning. Proceedings of the International Conference on Learning Representations, Toulon, France.
44. An approximate dynamic programming algorithm for large-scale fleet management: A case application10.1287/trsc.1080.0238 ↗
45. An approximate dynamic programming approach for the vehicle routing problem with stochastic demands10.1016/j.ejor.2008.03.023 ↗
46. A unified framework for stochastic optimization10.1016/j.ejor.2018.07.014 ↗
47. A reinforcement learning approach to convoy scheduling on a contested transportation network10.1007/s11590-015-0875-6 ↗
48. Rollout policies for dynamic solutions to the multivehicle routing problem with stochastic demand and duration limits10.1287/opre.1120.1127 ↗
49. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). Openai gym. arXiv.
50. Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to Sequence Learning with Neural Networks. Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2 (NIPS’14), MIT Press. Available online: https://proceedings.neurips.cc/paper_files/paper/2014/file/a14ac55a4f27472c5d894ec1c3c743d2-Paper.pdf.
51. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. In ICLR 2015. arXiv.
52. Zoph, B., and Le, Q.V. (2016). Neural architecture search with reinforcement learning. arXiv.
53. Nowak, A., and Bruna, J. (2024, August 01). Divide and Conquer with Neural Networks. Available online: https://openreview.net/forum?id=Hy3_KuYxg.
54. Vinyals, O., Fortunato, M., and Jaitly, N. (2015). Pointer networks. Adv. Neural Inf. Process. Syst., 28, Available online: https://papers.nips.cc/paper_files/paper/2015/file/29921001f2f04bd3baee84a12e98098f-Paper.pdf.
55. Kool, W., Van Hoof, H., and Welling, M. (2018). Attention, learn to solve routing problems!. arXiv.
56. Deudon, M., Cournut, P., Lacoste, A., Adulyasak, Y., and Rousseau, L.M. (2018, January 26–29). Learning heuristics for the tsp by policy gradient. Proceedings of the Integration of Constraint Programming, Artificial Intelligence, and Operations Research: 15th International Conference, Delft, The Netherlands.10.1007/978-3-319-93031-2_12 ↗
57. Vinyals, O., Bengio, S., and Kudlur, M. (2015). Order matters: Sequence to sequence for sets. arXiv.
58. Deep reinforcement learning for solving the heterogeneous capacitated vehicle routing problem10.1109/TCYB.2021.3111082 ↗
59. A multi-agent reinforcement learning method with route recorders for vehicle routing in supply chain management10.1109/TITS.2022.3150151 ↗
60. Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., and Sun, L. (2022). Transformers in time series: A survey. arXiv.10.24963/ijcai.2023/759 ↗
61. The graph neural network model10.1109/TNN.2008.2005605 ↗
62. Graph attention networks
63. Graph neural network for traffic forecasting: A survey10.1016/j.eswa.2022.117921 ↗
64. He, S., and Shin, K.G. (2020). Towards fine-grained flow forecasting: A graph attention approach for bike sharing systems. Proc. Web Conf., 88–98.10.1145/3366423.3380097 ↗
65. Fang, X., Huang, J., Wang, F., Zeng, L., Liang, H., and Wang, H. (2020, January 6–10). Constgat: Contextual spatial-temporal graph attention network for travel time estimation at baidu maps. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, Virtual.10.1145/3394486.3403320 ↗
66. Attention-based graph neural networks: A survey10.1007/s10462-023-10577-2 ↗
67. A deep q-network for the beer game: Deep reinforcement learning for inventory optimization10.1287/msom.2020.0939 ↗
68. Obara, M., Kashiyama, T., and Sekimoto, Y. (2018, January 10–13). Deep reinforcement learning approach for train rescheduling utilizing graph theory. Proceedings of the 2018 IEEE International Conference on Big Data, Seattle, WA, USA.10.1109/BigData.2018.8622214 ↗
69. A scalable reinforcement learning algorithm for scheduling railway lines10.1109/TITS.2018.2829165 ↗
70. Guo, W., Atasoy, B., and Negenborn, R.R. (2022). Global synchromodal shipment matching problem with dynamic and stochastic travel times: A reinforcement learning approach. Ann. Oper. Res., 1–32.10.1007/s10479-021-04489-z ↗
71. Adi, T.N., Iskandar, Y.A., and Bae, H. (2020). Interterminal truck routing optimization using deep reinforcement learning. Sensors, 20.10.3390/s20205794 ↗
72. A review on reinforcement learning algorithms and applications in supply chain management10.1080/00207543.2022.2140221 ↗
73. Reinforcement learning for logistics and supply chain management: Methodologies, state of the art, and future opportunities10.1016/j.tre.2022.102712 ↗
74. Zong, Z., Feng, T., Xia, T., Jin, D., and Li, Y. (2021). Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey. arXiv.
75. Deep reinforcement learning for transportation network combinatorial optimization: A survey10.1016/j.knosys.2021.107526 ↗
76. Gokhale, A., Trasikar, C., Shah, A., Hegde, A., and Naik, S.R. (2019). A Reinforcement Learning Approach to Inventory Management. Advances in Artificial Intelligence and Data Engineering: Select Proceedings of AIDE, Springer.10.1007/978-981-15-3514-7_23 ↗
77. Li, X., Zhang, J., Bian, J., Tong, Y., and Liu, T.Y. (2019). A cooperative multi-agent reinforcement learning framework for resource balancing in complex logistics network. arXiv.
78. Deep reinforcement learning for inventory control: A roadmap10.1016/j.ejor.2021.07.016 ↗
79. Chen, J., Umrawal, A.K., Lan, T., and Aggarwal, V. (2021, January 2–13). DeepFreight: A Model-free Deep-reinforcement-learning-based Algorithm for Multi-transfer Freight Delivery. Proceedings of the International Conference on Automated Planning and Scheduling, Guangzhou, China.10.1609/icaps.v31i1.15998 ↗
80. Vera, J.M., and Abad, A.G. (2019, January 11–15). Deep reinforcement learning for routing a heterogeneous fleet of vehicles. Proceedings of the 2019 IEEE Latin American Conference on Computational Intelligence (LA-CCI), Guayaquil, Ecuador.10.1109/LA-CCI47412.2019.9037042 ↗
81. Liu, X., Hu, M., Peng, Y., and Yang, Y. (2022). Multi-Agent Deep Reinforcement Learning for Multi-Echelon Inventory Management, SSRN.10.2139/ssrn.4262186 ↗
82. Chen, Y., Qian, Y., Yao, Y., Wu, Z., Li, R., and Xu, Y. (2019). Can sophisticated dispatching strategy acquired by reinforcement learning?—A case study in dynamic courier dispatching system. arXiv.
83. Beeks, M., Afshar, R.R., Zhang, Y., Dijkman, R., van Dorst, C., and de Looijer, S. (2022, January 13–24). Deep reinforcement learning for a multi-objective online order batching problem. Proceedings of the International Conference on Automated Planning and Scheduling, Virtual.10.1609/icaps.v32i1.19829 ↗
84. Hierarchical reinforcement learning: A survey and open research challenges10.3390/make4010009 ↗
85. Machine learning for combinatorial optimization: A methodological tour d’horizon10.1016/j.ejor.2020.07.063 ↗
86. Biedenkapp, A., Bozkurt, H.F., Eimer, T., Hutter, F., and Lindauer, M. (2020). Dynamic algorithm configuration: Foundation of a new meta-algorithmic framework. ECAI 2020, IOS Press.
87. Ni, F., Hao, J., Lu, J., Tong, X., Yuan, M., and He, K. (2021, January 14–18). A multi-graph attributed reinforcement learning based optimization algorithm for large-scale hybrid flow shop scheduling problem. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery Data Mining, Virtual.10.1145/3447548.3467135 ↗
88. Kerkkamp, D., Bukhsh, Z.A., Zhang, Y., and Jansen, N. (2022, January 3–5). Grouping of Maintenance Actions with Deep Reinforcement Learning and Graph Convolutional Networks. Proceedings of the ICAART, Virtual.10.5220/0010907500003116 ↗
89. Popescu, T. (2024, September 20). Reinforcement Learning for Train Dispatching: A Study on the Possibility to Use Reinforcement Learning to Optimize Train Ordering and Minimize Train Delays in Disrupted Situations, inside the Rail Simulator OSRD. KTH, School of Electrical Engineering and Computer Science (EECS). Dissertation. Available online: https://www.diva-portal.org/smash/get/diva2:1702837/FULLTEXT01.pdf.
90. Zhu, H., Gupta, V., Ahuja, S.S., Tian, Y., Zhang, Y., and Jin, X. (2021, January 23–27). Network planning with deep reinforcement learning. Proceedings of the 2021 ACM SIGCOMM 2021 Conference, Virtual.10.1145/3452296.3472902 ↗
91. Flexible Job-Shop Scheduling via Graph Neural Network and Deep Reinforcement Learning10.1109/TII.2022.3189725 ↗
92. A novel solution to jsps based on long short-term memory and policy gradient algorithm10.2507/IJSIMM19-1-CO4 ↗
93. A deep reinforcement learning based solution for flexible job shop scheduling problem10.2507/IJSIMM20-2-CO7 ↗
94. Oren, J., Ross, C., Lefarov, M., Richter, F., Taitler, A., and Daniel, C. (2021, January 26–30). SOLO: Search online, learn offline for combinatorial optimization problems. Proceedings of the International Symposium on Combinatorial Search, Virtual.10.1609/socs.v12i1.18556 ↗
95. Weighted qmix: Expanding monotonic value function factorisation for deep multi-agent reinforcement learning
96. Chen, H., Li, Z., and Yao, Y. (2022, January 25–27). Multi-agent reinforcement learning for fleet management: A survey. Proceedings of the 2nd International Conference on Artificial Intelligence, Automation, and High-Performance Computing, AIAHPC 2022, Zhuhai, China.10.1117/12.2641877 ↗
97. Cooperative zone-based rebalancing of idle overhead hoist transportations using multi-agent reinforcement learning with graph representation learning
98. Pan, L., Cai, Q., Fang, Z., Tang, P., and Huang, L. (February, January 27). A deep reinforcement learning framework for rebalancing dockless bike sharing systems. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
99. HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand10.1109/TITS.2022.3191752 ↗
100. Solving the online batching problem using deep reinforcement learning10.1016/j.cie.2021.107221 ↗