journal article·2026-01-30Open access

Industrial Process Control Based on Reinforcement Learning: Taking Tin Smelting Parameter Optimization as an Example

Yingli Liu, Zheng Xiong, Haibin Yuan, Hang Yan, Ling Yang

Applied Sciences· Volume 16· Issue 3

Abstract

To address the issues of parameter setting, reliance on human experience, and the limitations of traditional model-driven control methods in handling complex nonlinear dynamics in the tin smelting industrial process, this paper proposes a data-driven control approach based on improved deep reinforcement learning (RL). Aiming to reduce the tin entrainment rate in smelting slag and CO emissions in exhaust gas, we construct a data-driven environment model with an 8-dimensional state space (including furnace temperature, pressure, gas composition, etc.) and an 8-dimensional action space (including lance parameters such as material flow, oxygen content, backpressure, etc.). We innovatively design a Dual-Action Discriminative Deep Deterministic Policy Gradient (DADDPG) algorithm. This method employs an online Actor network to simultaneously generate deterministic and exploratory random actions, with the Critic network selecting high-value actions for execution, consistently enhancing policy exploration efficiency. Combined with a composite reward function (integrating real-time Sn/CO content, their variations, and continuous penalty mechanisms for safety constraints), the approach achieves multi-objective dynamic optimization. Experiments based on real tin smelting production line data validate the environment model, with results demonstrating that the tin content in slag is reduced to between 3.5% and 4%, and CO content in exhaust gas is decreased to between 2000 and 2700 ppm.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Adaptive Dynamic Programming Control · Digital Transformation in Industry · Iron and Steelmaking Processes

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650093, China

Yunnan Key Laboratory of Computer Technologies Application, Kunming University of Science and Technology, Kunming 650500, China

Tin Branch of Yunnan Tin Company Limited, Gejiu 661017, China

Funding

Major Science and Technology Projects in Yunnan Province

References (41)

Show source-provided references
1. Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers10.1109/MCS.2012.2214134 ↗
2. Spielberg, S.P.K., Gopaluni, R.B., and Loewen, P.D. (2017, January 28–31). Deep reinforcement learning approaches for process control. Proceedings of the 2017 6th International Symposium on Advanced Control of Industrial Processes, Taiwan, Taipei.10.1109/ADCONIP.2017.7983780 ↗
3. Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems10.1109/TNNLS.2013.2281663 ↗
4. Integrated experimental and thermodynamic modeling study of slag–matte–metal equilibrium in the Pb–Fe–O–S–Si-(Al, Ca, Zn) systems at 1100–1200 °C10.1016/j.jmrt.2024.06.188 ↗
5. Experimental investigation and optimization of the effects of manufacturing parameters on geometric tolerances in additive manufacturing of AlSi10Mg alloy10.1007/s00170-024-14128-z ↗
6. Neural network model construction and analysis for simulating and predicting hazardous element allocation in copper smelting key links10.1016/j.jhazmat.2025.138725 ↗
7. Quality prediction for aluminum diecasting process based on shallow neural network and data feature selection technique10.1016/j.cirpj.2021.04.001 ↗
8. Transductive transfer broad learning for cross-domain information exploration and multigrade soft sensor application10.1016/j.chemolab.2023.104778 ↗
9. Deep reinforcement learning for digital materials design10.1021/acsmaterialslett.1c00390 ↗
10. Real-time evolution for on-line optimization of continuous processes10.1021/ie010464l ↗
11. An intelligent switching control for a mixed separation thickener process10.1016/j.conengprac.2016.07.007 ↗
12. Stochastic choice of basis functions in adaptive function approximation and the functional-link net10.1109/72.471375 ↗
13. Reinforcement learning for industrial process control: A case study in flatness control in steel industry10.1016/j.compind.2022.103748 ↗
14. Deep reinforcement learning for maintenance optimization of a scrap-based steel production line10.1016/j.ress.2024.110199 ↗
15. Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming10.1016/j.automatica.2012.05.049 ↗
16. Hierarchical granular computing-based model and its reinforcement structural learning for construction of long-term prediction intervals10.1109/TCYB.2020.2964011 ↗
17. Safe reinforcement learning for industrial optimal control: A case study from metallurgical industry10.1016/j.ins.2023.119684 ↗
18. Model-free emergency frequency control based on reinforcement learning10.1109/TII.2020.3001095 ↗
19. Deep reinforcement learning-based autonomous voltage control for power grid operations10.1109/TPWRS.2019.2941134 ↗
20. Fan, H., Zhu, L., Yao, C., Guo, Y., Jiao, X., and Jin, J. (2019, January 12–15). Deep reinforcement learning for energy efficiency optimization in wireless networks. Proceedings of the 2019 IEEE 4th International Conference on Cloud Computing and Big Data Analysis, Chengdu, China.10.1109/ICCCBDA.2019.8725683 ↗
21. Joint computing and caching in 5G-envisioned Internet of Vehicles: A deep reinforcement learning-based traffic control system10.1109/TITS.2020.2970276 ↗
22. Two-timescale coordinated voltage regulation for high renewable-penetrated active distribution networks considering hybrid devices10.1109/TII.2023.3308348 ↗
23. Reward shaping-based actor–critic deep reinforcement learning for residential energy management10.1109/TII.2022.3183802 ↗
24. Digital twin-driven smelting process management method for converter steelmaking10.1007/s10845-024-02366-7 ↗
25. IoT-based framework for digital twins in steel production: A case study of key parameter prediction and optimization for CSR10.1016/j.eswa.2024.123909 ↗
26. Offline constrained reinforcement learning for batch-to-batch optimization of cobalt oxalate synthesis process10.1016/j.cherd.2024.08.013 ↗
27. Machine learning approach for predicting tramp elements in the basic oxygen furnace based on the compiled steel scrap mix10.1038/s41598-025-86406-z ↗
28. Huang, W. (2001). Tin, Metallurgical Industry Press.
29. Song, X. (2011). Tin Metallurgy, Metallurgical Industry Press.
30. A Huber reward function-driven deep reinforcement learning solution for cart-pole balancing problem10.1007/s00521-022-07606-6 ↗
31. Model-based reinforcement learning for physical systems without velocity and acceleration measurements
32. Lukas, M., Tomicic, I., and Bernik, A. (2022). Anticheat system based on reinforcement learning agents in Unity. Information, 13.10.3390/info13040173 ↗
33. Ranaweera, M., and Mahmoud, Q.H. (2024). Deep Reinforcement Learning with Godot Game Engine. Electronics, 13.10.3390/electronics13050985 ↗
34. Game of drones: Multi-UAV pursuit-evasion game with online motion planning by deep reinforcement learning10.1109/TNNLS.2022.3146976 ↗
35. Graph learning-based generation of abstractions for reinforcement learning10.1007/s00521-023-08211-x ↗
36. Data-driven Offline Reinforcement Learning for HVAC-systems10.1016/j.energy.2022.125290 ↗
37. High-fidelity data-driven dynamics model for reinforcement learning-based control in HL-3 tokamak10.1038/s42005-025-02302-y ↗
38. Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 22–24). Deterministic policy gradient algorithms. Proceedings of the 31st International Conference on Machine Learning, Beijing, China.
39. Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10–15). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden. Available online: https://proceedings.mlr.press/v80/haarnoja18b.html.
40. Fujimoto, S., van Hoof, H., and Meger, D. (2018, January 10–15). Addressing function approximation error in actor-critic methods. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden. Available online: https://proceedings.mlr.press/v80/fujimoto18a.html.
41. Proximal policy optimization with policy feedback