Convolution and Long Short-Term Memory Hybrid Deep Neural Networks for Remaining Useful Life Prognostics
Source abstract
Abstract
Reliable prediction of remaining useful life (RUL) plays an indispensable role in prognostics and health management (PHM) by reason of the increasing safety requirements of industrial equipment. Meanwhile, data-driven methods in RUL prognostics have attracted widespread interest. Deep learning as a promising data-driven method has been developed to predict RUL due to its ability to deal with abundant complex data. In this paper, a novel scheme based on a health indicator (HI) and a hybrid deep neural network (DNN) model is proposed to predict RUL by analyzing equipment degradation. Explicitly, HI obtained by polynomial regression is combined with a convolutional neural network (CNN) and long short-term memory (LSTM) neural network to extract spatial and temporal features for efficacious prognostics. More specifically, valid data selected from the raw sensor data are transformed into a one-dimensional HI at first. Next, both the preselected data and HI are sequentially fed into the CNN layer and LSTM layer in order to extract high-level spatial features and long-term temporal dependency features. Furthermore, a fully connected neural network is employed to achieve a regression model of RUL prognostics. Lastly, validated with the aid of numerical and graphic results by an equipment RUL dataset from the Commercial Modular Aero-Propulsion System Simulation(C-MAPSS), the proposed scheme turns out to be superior to four existing models regarding accuracy and effectiveness.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Fault Diagnosis Techniques · Reliability and Maintenance Optimization · Fault Detection and Control Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China
Funding
National Natural Science Foundation of China · Natural Science Foundation of Hubei Province
References (55)
Show source-provided references
1. Long short-term memory for machine remaining life prediction10.1016/j.jmsy.2018.05.011 ↗
2. An indirect RUL prognosis for lithium-ion battery under vibration stress using Elman neural network10.1016/j.ijhydene.2019.03.101 ↗
3. A multidimensional hybrid intelligent method for gear fault diagnosis10.1016/j.eswa.2009.06.060 ↗
4. Coro, A., Macareno, L.M., Aguirrebeitia, J., and López de Lacalle, L.N. (2019). A Methodology to Evaluate the Reliability Impact of the Replacement of Welded Components by Additive Manufacturing Spare Parts. Metals, 9.10.3390/met9090932 ↗
5. Baysal, M.E., Sümbül, M.O., and Ekicioğlu, E. (2015, January 27–29). A total productive maintenance implementation in a manufacturing company operating in insulation sector in Turkey. Proceedings of the 2015 6th International Conference on Modeling, Simulation, and Applied Optimization (ICMSAO), Istanbul, Turkey.10.1109/ICMSAO.2015.7152205 ↗
6. A Review of Current Prognostics and Health Management System Related Standards
7. Time Series Prediction Techniques for Estimating Remaining Useful Lifetime of Cutting Tool Failure
8. Zheng, S., Ristovski, K., Farahat, A., and Gupta, C. (2017, January 19–21). Long Short-Term Memory Network for Remaining Useful Life estimation. Proceedings of the 2017 IEEE International Conference on Prognostics and Health Management (ICPHM), Dallas, TX, USA.10.1109/ICPHM.2017.7998311 ↗
9. A hybrid framework combining data-driven and model-based methods for system remaining useful life prediction10.1016/j.asoc.2016.03.013 ↗
10. A new remaining useful life estimation method for equipment subjected to intervention of imperfect maintenance activities10.1016/j.cja.2018.01.009 ↗
11. Overview of Remaining Useful Life Prediction Techniques in Through-life Engineering Services10.1016/j.procir.2014.02.006 ↗
12. Remaining useful life estimation—A review on the statistical data driven approaches10.1016/j.ejor.2010.11.018 ↗
13. Machinery health prognostics: A systematic review from data acquisition to RUL prediction10.1016/j.ymssp.2017.11.016 ↗
14. Basak, S., Sengupta, S., and Dubey, A. (2018). A Data-driven Prognostic Architecture for Online Monitoring of Hard Disks Using Deep LSTM Networks. arXiv.
15. Data-Driven Remaining Useful Life Prediction Considering Sensor Anomaly Detection and Data Recovery10.1109/ACCESS.2019.2914236 ↗
16. Srivastava, A.N., and Han, J. (2011). Machine Learn Ing and Knowledge Discovery for Engineering Systems Health Management, CRC Press.
17. Effects of tool deflection in the high-speed milling of inclined surfaces10.1007/s00170-003-1723-x ↗
18. Force and temperature modelling of bone milling using artificial neural networks10.1016/j.measurement.2017.10.051 ↗
19. Wu, W., Hu, J., and Zhang, J. (2007, January 23–25). Prognostics of Machine Health Condition using an Improved ARIMA-based Prediction method. Proceedings of the 2007 2nd IEEE Conference on Industrial Electronics and Applications, Harbin, China.10.1109/ICIEA.2007.4318571 ↗
20. An ARIMA-ANN Hybrid Model for Time Series Forecasting10.1002/sres.2179 ↗
21. An HMM and polynomial regression based approach for remaining useful life and health state estimation of cutting tools10.1016/j.cie.2018.05.017 ↗
22. Process Fault Prognosis Using Hidden Markov Model–Bayesian Networks Hybrid Model10.1021/acs.iecr.9b00524 ↗
23. A Data-Driven Failure Prognostics Method Based on Mixture of Gaussians Hidden Markov Models10.1109/TR.2012.2194177 ↗
24. Pang, B., Feng, W., Zhao, H., Li, W., and Chen, S. (2018, January 26–28). Research on Modeling Method of Life Prediction for Satellite Lithium Battery Based on SVR. Proceedings of the 2018 Prognostics and System Health Management Conference (PHM-Chongqing), Chongqing, China.10.1109/PHM-Chongqing.2018.00179 ↗
25. Failure and reliability prediction by support vector machines regression of time series data10.1016/j.ress.2011.06.006 ↗
26. Remaining useful life estimation based on nonlinear feature reduction and support vector regression10.1016/j.engappai.2013.02.006 ↗
27. Varanasi, J., and Tripathi, M.M. (2016, January 4–6). Artificial Neural Network based wind speed amp; power forecasting in US wind energy farms. Proceedings of the 2016 IEEE 1st International Conference on Power Electronics, Intelligent Control and Energy Systems (ICPEICES), Delhi, India.10.1109/ICPEICES.2016.7853622 ↗
28. Using artificial neural networks for the prediction of dimensional error on inclined surfaces manufactured by ball-end milling10.1007/s00170-015-7543-y ↗
29. Wu, D., Jennings, C., Terpenny, J., Gao, R., and Tirupatikumara, S. (2017). A Comparative Study on Machine Learning Algorithms for Smart Manufacturing: Tool Wear Prediction Using Random Forests. J. Manuf. Sci. Eng. Trans. ASME, 139.10.1115/1.4036350 ↗
30. Modeling of electrical discharge machining process using back propagation neural network and multi-objective optimization using non-dominating sorting genetic algorithm-II10.1016/j.jmatprotec.2006.12.030 ↗
31. Deep learning10.1038/nature14539 ↗
32. Wang, J., Zhuang, J., Duan, L., and Cheng, W. (2016, January 1–3). A multi-scale convolution neural network for featureless fault diagnosis. Proceedings of the 2016 International Symposium on Flexible Automation (ISFA), Cleveland, OH, USA.10.1109/ISFA.2016.7790137 ↗
33. Prediction of Bearing Remaining Useful Life With Deep Convolution Neural Network10.1109/ACCESS.2018.2804930 ↗
34. Navathe, S.B., Wu, W., Shekhar, S., Du, X., Wang, X.S., and Xiong, H. (2016). Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life. Database Systems for Advanced Applications, Springer.10.1007/978-3-319-32025-0 ↗
35. Zhang, J., Wang, J., He, L., Li, Z., and Yu, P.S. (2018, January 17–20). Layerwise Perturbation-Based Adversarial Training for Hard Drive Health Degree Prediction. Proceedings of the 2018 IEEE International Conference on Data Mining (ICDM), Singapore.10.1109/ICDM.2018.00197 ↗
36. A Recurrent Neural Network Based Health Indicator for Remaining Useful Life Prediction of Bearings10.1016/j.neucom.2017.02.045 ↗
37. Wang, J., Yu, L.C., Lai, K.R., and Zhang, X.J. (2016, January 7–12). Dimensional Sentiment Analysis Using a Regional CNN-LSTM Model. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.10.18653/v1/P16-2037 ↗
38. Sainath, T.N., Vinyals, O., Senior, A., and Sak, H. (2015, January 19–24). Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Australia.10.1109/ICASSP.2015.7178838 ↗
39. Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features10.1109/ACCESS.2017.2778011 ↗
40. Automated diagnosis of arrhythmia using combination of CNN and LSTM techniques with variable length heart beats10.1016/j.compbiomed.2018.06.002 ↗
41. Zhao, R., Yan, R., Wang, J., and Mao, K. (2017). Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks. Sensors, 17.10.3390/s17020273 ↗
42. Yue, G., Ping, G., and Lanxin, L. (2018, January 22–24). An End-to-End model based on CNN-LSTM for Industrial Fault Diagnosis and Prognosis. Proceedings of the 2018 International Conference on Network Infrastructure and Digital Content (IC-NIDC), Guiyang, China.10.1109/ICNIDC.2018.8525759 ↗
43. Predicting residential energy consumption using CNN-LSTM neural networks10.1016/j.energy.2019.05.230 ↗
44. Saxena, A., Goebel, K., Simon, D., and Eklund, N. (2008, January 6–9). Damage propagation modeling for aircraft engine run-to-failure simulation. Proceedings of the 2008 International Conference on Prognostics and Health Management, Denver, CO, USA.10.1109/PHM.2008.4711414 ↗
45. Volponi, A.J., DePold, H., Ganguli, R., and Daguang, C. (2003). The Use of Kalman Filter and Neural Network Methodologies in Gas Turbine Performance Diagnostics: A Comparative Study. J. Eng. Gas Turbines Power, 125.10.1115/1.1419016 ↗
46. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position10.1007/BF00344251 ↗
47. Chen, Z., Li, C., and Sanchez, R. (2015). Gearbox Fault Identification and Classification with Convolutional Neural Networks. Shock Vib.10.1155/2015/390134 ↗
48. A CNN-BiLSTM Model for Document-Level Sentiment Analysis10.3390/make1030048 ↗
49. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
50. Hwan Lee, J., and Kim, H. (2019, January 24–26). Empirical Investigation of Stale Value Tolerance on Parallel RNN Training. Proceedings of the 2019 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), Madison, WI, USA.10.1109/ISPASS.2019.00029 ↗
51. Long short-term memory neural network for traffic speed prediction using remote microwave sensor data10.1016/j.trc.2015.03.014 ↗
52. Chang, Z., Zhang, Y., and Chen, W. (2018, January 23–25). Effective Adam-Optimized LSTM Neural Network for Electricity Price Forecasting. Proceedings of the 2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS), Beijing, China.10.1109/ICSESS.2018.8663710 ↗
53. Guan, Y., Dy, J.G., and Jordan, M.I. (July, January 28). A Unified Probabilistic Model for Global and Local Unsupervised Feature Selection. Proceedings of the 28th International Conference on Machine Learning (ICML), Bellevue, WA, USA.
54. Patro, S.G.K., and Sahu, K.K. (2015). Normalization: A Preprocessing Stage. arXiv.10.17148/IARJSET.2015.2305 ↗
55. Support vector regression