journal article·2024-09-26Open access

DeepESN Neural Networks for Industrial Predictive Maintenance through Anomaly Detection from Production Energy Data

Andrea Bonci, Luca Fredianelli, Renat Kermenov, Lorenzo Longarini, Sauro Longhi, Geremia Pompei, Mariorosario Prist, Carlo Verdini

Applied Sciences· Volume 14· Issue 19

Abstract

Optimizing energy consumption is an important aspect of industrial competitiveness, as it directly impacts operational efficiency, cost reduction, and sustainability goals. In this context, anomaly detection (AD) becomes a valuable methodology, as it supports maintenance activities in the manufacturing sector, allowing for early intervention to prevent energy waste and maintain optimal performance. Here, an AD-based method is proposed and studied to support energy-saving predictive maintenance of production lines using time series acquired directly from the field. This paper proposes a deep echo state network (DeepESN)-based method for anomaly detection by analyzing energy consumption data sets from production lines. Compared with traditional prediction methods, such as recurrent neural networks with long short-term memory (LSTM), although both models show similar time series trends, the DeepESN-based method studied here appears to have some advantages, such as timelier error detection and higher prediction accuracy. In addition, the DeepESN-based method has been shown to be more accurate in predicting the occurrence of failure. The proposed solution has been extensively tested in a real-world pilot case consisting of an automated metal filter production line equipped with industrial smart meters to acquire energy data during production phases; the time series, composed of 88 variables associated with energy parameters, was then processed using the techniques introduced earlier. The results show that our method enables earlier error detection and achieves higher prediction accuracy when running on an edge device.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Energy Load and Power Forecasting · Anomaly Detection Techniques and Applications · Fault Detection and Control Systems

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Information Engineering (DII), Polytechnic University of Marche, 60131 Ancona, Italy

Institute of Chemical and Physical Processes of National Research Council, 56124 Pisa, Italy

Syncode Scarl, 93900 Fermo, Italy

Funding

the European project Horizon Europe “EDIH4Marche” (European Digital Innovation Hub for Marche), Call DIGITAL-2021-EDIH-01

References (74)

Show source-provided references
1. Climate change and COP26: Are digital technologies and information management part of the problem or the solution? An editorial reflection and call to action
2. Resilient adaptive control based on renewal particle swarm optimization to improve production system energy efficiency10.1016/j.jmsy.2018.12.007 ↗
3. Energy saving opportunity analysis of automotive serial production systems
4. Trends in fault diagnosis for electrical machines: A review of diagnostic techniques10.1109/MIE.2013.2287651 ↗
5. Bonci, A., Longhi, S., and Nabissi, G. (2021, January 8–9). Fault Diagnosis in a belt-drive system under non-stationary conditions. An industrial case study. Proceedings of the 2021 IEEE Workshop on Electrical Machines Design, Control and Diagnosis (WEMDCD), Modena, Italy.10.1109/WEMDCD51469.2021.9425680 ↗
6. Condition monitoring and fault diagnosis of electrical motors—A review10.1109/TEC.2005.847955 ↗
7. Development of an energy management system—Case study of a Serbian car manufacturer10.1016/j.enconman.2010.06.014 ↗
8. Carbon element flow analysis and CO2 emission reduction in iron and steel works10.1016/j.jclepro.2017.10.211 ↗
9. Selection of cutting inserts in dry machining for reducing energy consumption and CO2 emissions10.3390/en81112362 ↗
10. Bonci, A., Di Biase, A., Dragoni, A.F., Longhi, S., Sernani, P., and Zega, A. (2022, January 6–9). Machine learning for monitoring and predictive maintenance of cutting tool wear for clean-cut machining machines. Proceedings of the IEEE 27th International Conference on Emerging Technologies and Factory Automation (ETFA), Stuttgart, Germany.10.1109/ETFA52439.2022.9921571 ↗
11. Sustainable machining: Selection of optimum turning conditions based on minimum energy considerations10.1016/j.jclepro.2010.01.025 ↗
12. Optimization of process parameters using a response surface method for minimizing power consumption in the milling of carbon steel10.1016/j.jclepro.2013.10.025 ↗
13. A review on energy saving strategies in industrial sector10.1016/j.rser.2010.09.003 ↗
14. An exploratory study on the relationship of Overall Equipment Effectiveness (OEE) variables and CO2 emissions10.1016/j.promfg.2019.07.050 ↗
15. Bonci, A., Stadnicka, D., and Longhi, S. (2022, January 16–19). The Overall Labour Effectiveness to Improve Competitiveness and Productivity in Human-Centered Manufacturing. Proceedings of the International Scientific-Technical Conference Manufacturing, Poznan, Poland.10.1007/978-3-030-99310-8_12 ↗
16. A Predictive Maintenance System for Epitaxy Processes Based on Filtering and Prediction Techniques10.1109/TSM.2012.2209131 ↗
17. Mission reliability-driven risk-based predictive maintenance approach of multistate manufacturing system10.1016/j.ress.2023.109273 ↗
18. Ariyo, A.A., Adewumi, A.O., and Ayo, C.K. (2014, January 26–28). Stock Price Prediction Using the ARIMA Model. Proceedings of the 16th International Conference on Computer Modelling and Simulation, Cambridge, UK.10.1109/UKSim.2014.67 ↗
19. Coupling HEC-HMS with Atmospheric Models for Prediction of Watershed Runoff10.1061/(ASCE)1084-0699(2002)7:4(312) ↗
20. Alharbi, F.R., and Csala, D. (2022). A Seasonal Autoregressive Integrated Moving Average with Exogenous Factors (SARIMAX) Forecasting Model-Based Time Series Approach. Inventions, 7.10.3390/inventions7040094 ↗
21. Prognostic modelling options for remaining useful life estimation by industry10.1016/j.ymssp.2010.11.018 ↗
22. Predicting the need for vehicle compressor repairs using maintenance records and logged vehicle data10.1016/j.engappai.2015.02.009 ↗
23. Machine learning based concept drift detection for predictive maintenance10.1016/j.cie.2019.106031 ↗
24. Haibin, C., Tan, P.N., Potter, C., and Klooster, S. (May, January 30). Detection and characterization of anomalies in multivariate time series. Proceedings of the 2009 SIAM International Conference on Data Mining, Sparks, Nevada.
25. Seyr, H., Muskulus, M., and Klooster, S. (2019). Use of Markov Decision Processes in the Evaluation of Corrective Maintenance Scheduling Policies for Offshore Wind Farms. J. Energies, 12.10.3390/en12152993 ↗
26. A dynamic opportunistic maintenance model to maximize energy-based availability while reducing the life cycle cost of wind farms
27. An optimal integrated production and maintenance strategy for a multi-wind turbines system10.1080/00207543.2019.1680897 ↗
28. Optimal Predictive Maintenance Policy for an Ocean Wave Farm10.1109/TSTE.2018.2877437 ↗
29. Energy-Oriented Maintenance Decision-Making for Sustainable Manufacturing Based on Energy Saving Window10.1115/1.4038996 ↗
30. Threaded ensembles of autoencoders for stream learning10.1111/coin.12146 ↗
31. Yu, K., Shi, W., and Santoro, N. (2020). Designing a Streaming Algorithm for Outlier Detection in Data Mining—An Incremental Approach. J. Sensors, 20.10.3390/s20051261 ↗
32. Song, L., Liang, H., and Zheng, T. (2019, January 3–5). Real-Time Anomaly Detection Method for Space Imager Streaming Data Based on HTM Algorithm. Proceedings of the 2019 IEEE 19th International Symposium on High Assurance Systems Engineering (HASE), Hangzhou, China.10.1109/HASE.2019.00015 ↗
33. An anomaly detection framework for dynamic systems using a Bayesian hierarchical framework10.1016/j.apenergy.2019.02.025 ↗
34. Kroll, B., Schaffranek, D., Schriegel, S., and Niggemann, O. (2014, January 16–19). System modeling based on machine learning for anomaly detection and predictive maintenance in industrial plants. Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA), Barcelona, Spain.10.1109/ETFA.2014.7005202 ↗
35. Bonci, A., De Amicis, R., Longhi, S., Lorenzoni, E., and Scala, G.A. (2016, January 13–15). Motorcycle’s lateral stability issues: Comparison of methods for dynamic modelling of roll angle. Proceedings of the 2016 20th International Conference on System Theory, Control and Computing (ICSTCC), Sinaia, Romania.10.1109/ICSTCC.2016.7790733 ↗
36. Towards an All-Wheel Drive Motorcycle: Dynamic Modeling and Simulation10.1109/ACCESS.2020.3002685 ↗
37. Kermenov, R., Nabissi, G., Longhi, S., and Bonci, A. (2023). Anomaly Detection and Concept Drift Adaptation for Dynamic Systems: A General Method with Practical Implementation Using an Industrial Collaborative Robot. Sensors, 23.10.3390/s23063260 ↗
38. Patti, G., Alderisi, G., and Bello, L.L. (2015, January 8–11). SchedWiFi: An Innovative Approach to support Scheduled Traffic in Ad-hoc Industrial IEEE 802.11 networks. Proceedings of the 2015 IEEE 20th Conference on Emerging Technologies and Factory Automation (ETFA), Luxembourg.10.1109/ETFA.2015.7301460 ↗
39. Execution Time of Optimal Controls in Hard Real Time, a Minimal Execution Time Solution for Nonlinear SDRE10.1109/ACCESS.2020.3019776 ↗
40. Manaswi, N.K., and John, S. (2018). Deep Learning with Applications Using Python, Apress.10.1007/978-1-4842-3516-4 ↗
41. Wang, Z., Chen, Z., Ni, J., Liu, H., Chen, H., and Tang, J. (2021, January 14–18). Multi-scale one-class recurrent neural networks for discrete event sequence anomaly detection. Proceedings of the 27th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Virtual Event.10.1145/3447548.3467125 ↗
42. Kingmaand, D.P., and Welling, M. (2014). Auto-encoding variational Bayes. arXiv.
43. Unsupervised anomaly detection of industrial robots using sliding-window convolutional variational autoencoder10.1109/ACCESS.2020.2977892 ↗
44. Integration of deep feature extraction and ensemble learning for outlier detection10.1016/j.patcog.2019.01.002 ↗
45. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial networks. Adv. Neural Inf. Process. Syst., 27.
46. Akcay, S., Atapour-Abarghouei, A., and Breckon, T.P. (2018, January 2–6). Ganomaly: Semisupervised anomaly detection via adversarial training. Proceedings of the Asian Conference on Computer Vision, Perth, Australia.10.1007/978-3-030-20893-6_39 ↗
47. Zenati, H., Foo, C.S., Lecouat, B., Manek, G., and Chandrasekhar, V.R. (2018). Efficient ganbased anomaly detection. arXiv.
48. Donahue, J., Krähenbühl, P., and Darrell, T. (2016). Adversarial feature learning. arXiv.
49. Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
50. Schmidt, R.M. (2019). Recurrent Neural Networks (RNNs): A gentle Introduction and Overview. arXiv.
51. An experimental unification of reservoir computing methods10.1016/j.neunet.2007.04.003 ↗
52. Backpropagation through time: What it does and how to do it10.1109/5.58337 ↗
53. A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures10.1162/neco_a_01199 ↗
54. Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network10.1016/j.physd.2019.132306 ↗
55. An efficient gradient-based algorithm for on-line training of recurrent network trajectories10.1162/neco.1990.2.4.490 ↗
56. Hecht-Nielsen, R. (1992). Theory of the Backpropagation Neural Network, Academic Press.
57. Gallicchio, C., and Micheli, A. (2017). Deep Echo State Network (DeepESN): A Brief Survey. arXiv.10.1109/IJCNN.2018.8489464 ↗
58. Deep learning in neural networks: An overview10.1016/j.neunet.2014.09.003 ↗
59. Deep reservoir computing: A critical experimental analysis10.1016/j.neucom.2016.12.089 ↗
60. Jaeger, H. (2001). Short Term Memory in Echo State Networks, GMD Forschungszentrum Informationstechnik.
61. Optimization and applications of echo state networks with leaky-integrator neurons10.1016/j.neunet.2007.04.016 ↗
62. Echo state property of deep reservoir computing networks10.1007/s12559-017-9461-9 ↗
63. Gallicchio, C., and Micheli, A. (2018, January 8–13). Why Layering in Recurrent Neural Networks? A DeepESN Survey. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.10.1109/IJCNN.2018.8489368 ↗
64. Ritchie, H., Rosado, P., and Roser, M. (2024, September 16). Energy. Available online: https://ourworldindata.org/.
65. Prist, M., Longhi, S., Monteriù, A., Giuggioloni, F., and Freddi, A. (2015, January 14–17). An integrated simulation environment for Wireless Sensor Networks. Proceedings of the IEEE 16th International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM), Boston, MA, USA.10.1109/WoWMoM.2015.7158177 ↗
66. Grisostomi, M., Ciabattoni, L., Prist, M., Ippoliti, G., and Longhi, S. (2014, January 11–14). Application of a wireless sensor networks and Web2Py architecture for factory line production monitoring. Proceedings of the IEEE 11th International Multi-Conference on Systems, Signals & Devices (SSD14), Castelldefels-Barcelona, Spain.10.1109/SSD.2014.6808882 ↗
67. (2023, December 04). Seneca S604 Smart Meter Portal. Available online: https://www.seneca.it/media/4166/power_1912eng_r1.pdf.
68. Analysis of the Effect of Data Scaling on the Performance of the Machine Learning Algorithm for Plant Identification10.29207/resti.v4i1.1517 ↗
69. Effect of normalization methods on the performance of supervised learning algorithms applied to HTSeq-FPKM-UQ data sets: 7SK RNA expression as a predictor of survival in patients with colon adenocarcinoma10.1093/bib/bbx153 ↗
70. Ahsan, M.M., Mahmud, M.A.P., Saha, P.K., Gupta, K.D., and Siddique, Z. (2021). Effect of Data Scaling Methods on Machine Learning Algorithms and Model Performance. J. Technol., 9.10.3390/technologies9030052 ↗
71. Echo State Networks: Novel reservoir selection and hyperparameter optimization model for time series forecasting10.1016/j.neucom.2023.126317 ↗
72. HELP: An LSTM-based approach to hyperparameter exploration in neural network learning10.1016/j.neucom.2020.12.133 ↗
73. Global convergence of the trial-and-error method for the traffic-restraint congestion-pricing scheme with day-to-day flow dynamics10.1016/j.trc.2016.06.009 ↗
74. On the complexity of trial and error for constraint satisfaction problems10.1016/j.jcss.2017.07.005 ↗