journal article·2025-10-22Open access

Advancing LightGBM with data augmentation for predicting the residual strength of corroded pipelines

Qiankun Wang, Hongfang Lu, Fan Li, Y. Frank Cheng

npj Materials Degradation· Volume 9· Issue 1

Abstract

Machine learning methods have been widely applied in predicting the residual strength of corroded pipelines due to their powerful predictive capabilities. However, the effective application of these techniques is constrained by the limited availability of high-quality data, as traditional pipeline burst tests are both costly and time-consuming. This study addresses the challenge of data limitations by applying and comparing three advanced data augmentation models—Tabular Variational Autoencoder (TVAE), Copula Generative Adversarial Network (CopulaGAN), and conditional tabular generative adversarial network (CTGAN)—to enhance the corroded pipeline dataset. The augmented datasets were used to train a LightGBM model for residual strength prediction. Among the three, the CopulaGAN-LightGBM data augmentation yielded the best improvement, increasing the model’s R 2 by 4.46%. Additionally, SHapley Additive exPlanations (SHAP) analysis was conducted on the CopulaGAN-LightGBM model to interpret feature importance, identifying wall thickness, defect depth, and pipe diameter as the most influential factors affecting residual strength. Finally, a practical online platform implementing the proposed model has been developed to enable real-time residual strength prediction. The results demonstrate that combining LightGBM with effective data augmentation techniques provides a reliable solution to overcome data limitations in pipeline corrosion assessment.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Structural Integrity and Reliability Analysis · Corrosion Behavior and Inhibition · Non-Destructive Testing Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (43)

Show source-provided references
1. Li, X., Jia, R., Zhang, R., Yang, S. & Chen, G. A KPCA-BRANN based data-driven approach to model corrosion degradation of subsea oil pipelines. Reliab. Eng. Syst. Saf. 219, 108231 (2022).10.1016/j.ress.2021.108231 ↗
2. Lu, H. & Cheng, Y. F. Detecting urban gas pipeline leaks using a vehicle–canine collaboration strategy. Nat. Cities 2, 281–282 (2025).10.1038/s44284-025-00197-y ↗
3. Lu, H., Xi, D., Xiang, Y., Su, Z. & Cheng, Y. F. Vehicle-canine collaboration for urban pipeline methane leak detection. Nat. Cities 2, 336–343 (2025).10.1038/s44284-024-00183-w ↗
4. Shaik, N. B., Jongkittinarukorn, K., Benjapolakul, W. & Bingi, K. A novel neural network-based framework to estimate oil and gas pipelines life with missing input parameters. Sci. Rep. 14, 4511 (2024).10.1038/s41598-024-54964-3 ↗
5. Shaik, N. B. et al. Recurrent neural network-based model for estimating the life condition of a dry gas pipeline. Process Saf. Environ. Prot. 164, 639–650 (2022).10.1016/j.psep.2022.06.047 ↗
6. Xu, L. et al. The research progress and prospect of data mining methods on corrosion prediction of oil and gas pipelines. Eng. Fail. Anal. 144, 106951 (2023).10.1016/j.engfailanal.2022.106951 ↗
7. Kumari, P., Halim, S. Z., Kwon, J. S.-I. & Quddus, N. An integrated risk prediction model for corrosion-induced pipeline incidents using artificial neural network and Bayesian analysis. Process Saf. Environ. Prot. 167, 34–44 (2022).10.1016/j.psep.2022.07.053 ↗
8. Soomro, A. A. et al. Analysis of machine learning models and data sources to forecast burst pressure of petroleum corroded pipelines: a comprehensive review. Eng. Fail. Anal. 155, 107747 (2024).10.1016/j.engfailanal.2023.107747 ↗
9. Lu, H., Xu, Z.-D., Iseley, T. & Matthews, J. C. Novel data-driven framework for predicting residual strength of corroded pipelines. J. Pipeline Syst. Eng. Pract. 12, 04021045 (2021).10.1061/(ASCE)PS.1949-1204.0000587 ↗
10. Wu, T., Miao, X. & Song, F. Residual strength prediction of corroded pipelines based on physics-informed machine learning and domain generalization. Npj Mater. Degrad. 9, 12 (2025).10.1038/s41529-025-00561-2 ↗
11. Shaik, N. B., Pedapati, S. R. & Dzubir, F. A. B. A. Remaining useful life prediction of a piping system using artificial neural networks: a case study. Ain Shams Eng. J. 13, 101535 (2022).10.1016/j.asej.2021.06.021 ↗
12. Lu, H., Iseley, T., Matthews, J., Liao, W. & Azimi, M. An ensemble model based on relevance vector machine and multi-objective salp swarm algorithm for predicting burst pressure of corroded pipelines. J. Pet. Sci. Eng. 203, 108585 (2021).10.1016/j.petrol.2021.108585 ↗
13. Su, Y., Li, J., Yu, B., Zhao, Y. & Yao, J. Fast and accurate prediction of failure pressure of oil and gas defective pipelines using the deep learning model. Reliab. Eng. Syst. Saf. 216, 108016 (2021).10.1016/j.ress.2021.108016 ↗
14. Li, S., Zhang, Z., Qian, H., Wang, H. & Fan, F. Research on remaining bearing capacity evaluation method for corroded pipelines with complex shaped defects. Ocean Eng. 296, 116805 (2024).10.1016/j.oceaneng.2024.116805 ↗
15. Shuai, Y. et al. A novel framework for predicting the burst pressure of energy pipelines with clustered corrosion defects. Thin-Walled Struct. 205, 112413 (2024).10.1016/j.tws.2024.112413 ↗
16. Wang, Q. & Lu, H. Machine learning methods for predicting residual strength in corroded oil and gas steel pipes. Npj Mater. Degrad. 9, 30 (2025).10.1038/s41529-025-00573-y ↗
17. Xiao, R., Zayed, T., Meguid, M. A. & Sushama, L. Predicting failure pressure of corroded gas pipelines: A data-driven approach using machine learning. Process Saf. Environ. Prot. 184, 1424–1441 (2024).10.1016/j.psep.2024.02.051 ↗
18. Ma, H. et al. A new hybrid approach model for predicting burst pressure of corroded pipelines of gas and oil. Eng. Fail. Anal. 149, 107248 (2023).10.1016/j.engfailanal.2023.107248 ↗
19. Wang, Q. & Lu, H. A novel stacking ensemble learner for predicting residual strength of corroded pipelines. Npj Mater. Degrad. 8, 87 (2024).10.1038/s41529-024-00508-z ↗
20. Miao X., Zhao H. Novel method for residual strength prediction of defective pipelines based on HTLBO-DELM model. Reliabil. Eng. Syst. Saf. 237, 109369 (2023).10.1016/j.ress.2023.109369 ↗
21. Benjamin, A. C., Freire, J. L. F., Vieira, R. D., Diniz, J. L. C. & de Andrade, E. Q. Burst tests on pipeline containing interacting corrosion defects. In: 24th International Conference on Offshore Mechanics and Arctic Engineering (ASME, 2005).10.1115/OMAE2005-67059 ↗
22. Liu, X. et al. An ANN-based failure pressure prediction method for buried high-strength pipes with stray current corrosion defect. Energy Sci. Eng. 8, 248–259 (2020).10.1002/ese3.522 ↗
23. Lo, M., Karuppanan, S. & Ovinis, M. ANN- and FEA-based assessment equation for a corroded pipeline with a single corrosion defect. J. Mar. Sci. Eng. 10, 476 (2022).10.3390/jmse10040476 ↗
24. Zhao, X.-Y., Chen, J.-X., Chen, G.-M., Xu, J.-J. & Zhang, L.-W. Prediction of ultimate condition of FRP-confined recycled aggregate concrete using a hybrid boosting model enriched with tabular generative adversarial networks. Thin-Walled Struct. 182, 110318 (2023).10.1016/j.tws.2022.110318 ↗
25. Marani, A., Jamali, A. & Nehdi, M. L. Predicting ultra-high-performance concrete compressive strength using tabular generative adversarial networks. Materials 13, 4757 (2020).10.3390/ma13214757 ↗
26. Woldesellasse, H. & Tesfamariam, S. Data augmentation using conditional generative adversarial network (cGAN): application for prediction of corrosion pit depth and testing using neural network. J. Pipeline Sci. Eng. 3, 100091 (2023).10.1016/j.jpse.2022.100091 ↗
27. Ma, H. et al. Data augmentation of a corrosion dataset for defect growth prediction of pipelines using conditional tabular generative adversarial networks. Materials 17, 1142 (2024).10.3390/ma17051142 ↗
28. Soomro, A. A. et al. Data augmentation using SMOTE technique: application for prediction of burst pressure of hydrocarbons pipeline using supervised machine learning models. Results Eng. 24, 103233 (2024).10.1016/j.rineng.2024.103233 ↗
29. Qin, G. & Cheng, Y. F. A review on defect assessment of pipelines: principles, numerical solutions, and applications. Int. J. Press. Vessels Pip. 191, 104329 (2021).10.1016/j.ijpvp.2021.104329 ↗
30. Qin, G., Zhang, C., Wang, B., Ni, P. & Wang, Y. An interpretable machine learning model for failure pressure prediction of blended hydrogen natural gas pipelines containing a crack-in-dent defect. Energy 320, 135401 (2025).10.1016/j.energy.2025.135401 ↗
31. Ben Seghier, M. E. A., Mohamed, O. A. & Ouaer, H. Machine learning-based Shapley additive explanations approach for corroded pipeline failure mode identification. Structures 65, 106653 (2024).10.1016/j.istruc.2024.106653 ↗
32. Amaya-Gómez, R., Munoz Giraldo, F., Schoefs, F., Bastidas-Arteaga, E. & Sanchez-Silva, M. Recollected burst tests of experimental and FEM corroded pipelines (Mendeley Data, 2019).
33. Inan, M. S. K., Hossain, S. & Uddin, M. N. Data augmentation guided breast cancer diagnosis and prognosis using an integrated deep-generative framework based on breast tumor’s morphological information. Inform. Med. Unlocked 37, 101171 (2023).10.1016/j.imu.2023.101171 ↗
34. Xu, L., Skoularidou, M., Cuesta-Infante, A. & Veeramachaneni, K. Modeling tabular data using conditional GAN. In: 33rd Conference on Neural Information Processing Systems (NeurIPS, 2019).
35. Goodfellow, I. et al. Generative adversarial networks. Commun. Acm 63, 139–144 (2020).10.1145/3422622 ↗
36. He, G., Zhao, Y. & Yan, C. Application of tabular data synthesis using generative adversarial networks on machine learning-based multiaxial fatigue life prediction. Int. J. Press. Vessels Pip. 199, 104779 (2022).10.1016/j.ijpvp.2022.104779 ↗
37. Chia, M. Y., Koo, C. H., Huang, Y. F., Di Chan, W. & Pang, J. Y. Artificial intelligence generated synthetic datasets as the remedy for data scarcity in water quality index estimation. Water Resour. Manag. 37, 6183–6198 (2023).10.1007/s11269-023-03650-6 ↗
38. Zeng, S. et al. Prediction of compressive strength of FRP-confined concrete using machine learning: a novel synthetic data driven framework. J. Build. Eng. 94, 109918 (2024).10.1016/j.jobe.2024.109918 ↗
39. Marani, A. & Nehdi, M. L. Predicting shear strength of FRP-reinforced concrete beams using novel synthetic data driven deep learning. Eng. Struct. 257, 114083 (2022).10.1016/j.engstruct.2022.114083 ↗
40. Chen, J. et al. An error-corrected deep Autoformer model via Bayesian optimization algorithm and secondary decomposition for photovoltaic power prediction. Appl. Energy 377, 124738 (2025).10.1016/j.apenergy.2024.124738 ↗
41. Ke, G. et al. LightGBM: a highly efficient gradient boosting decision tree. In: 31st Annual Conference on Neural Information Processing Systems (NIPS, 2017).
42. Janizadeh, S. et al. Advancing the LightGBM approach with three novel nature-inspired optimizers for predicting wildfire susceptibility in Kaua’i and Moloka’i Islands, Hawaii. Expert Syst. Appl. 258, 124963 (2024).10.1016/j.eswa.2024.124963 ↗
43. Patki, N., Wedge, R., Veeramachaneni, K. & IEEE. The synthetic data vault. In: 3rd IEEE/ACM International Conference on Data Science and Advanced Analytics (DSAA, 2016).10.1109/DSAA.2016.49 ↗