Anomaly Detection of Metallurgical Energy Data Based on iForest-AE
Source abstract
Abstract
With the proliferation of the Internet of Things, a large amount of data is generated constantly by industrial systems, corresponding in many cases to critical tasks. It is particularly important to detect abnormal data to ensure the accuracy of data. Aiming at the problem that the training data are contaminated with anomalies in autoencoder-based anomaly detection, which makes it difficult to distinguish abnormal data from normal data, this paper proposes a data anomaly detection method that combines an isolated forest (iForest) and autoencoder algorithm. In this method (iForest-AE), the iForest algorithm was used to calculate the anomaly score of energy data, and the data with a lower anomaly score were selected for model training. After the test data passed through the autoencoder trained by normal data, the data whose reconstruction error was larger than the threshold were determined as an anomaly. Experiment results on the electricity consumption dataset showed that the iForest-AE method achieved an F1 score of 0.981, which outperformed other detection methods, and a significant advantage in anomaly detection.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Occupational Health and Safety Research · Fault Detection and Control Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Metallurgical and Energy Engineering, Kunming University of Science and Technology, Kunming 650093, China
Chuxiong Dianzhong Nonferrous Metals Co., Ltd., Chuxiong 675000, China
Funding
Yunnan Major Scientific and Technological Projects
References (30)
Show source-provided references
1. An empirical study on green environmental system certification affects financing cost of high energy consumption enterprises-taking metallurgical enterprises as an example10.1016/j.jclepro.2019.118848 ↗
2. Marine environment and maritime safety assessment using Port State Control database10.1016/j.chemosphere.2022.135245 ↗
3. Profiling Malaysian ship registration and seafarers for streamlining future Malaysian shipping governance10.1080/18366503.2021.1878981 ↗
4. A survey of machine learning methods applied to anomaly detection on drinking-water quality data10.1080/1573062X.2019.1637002 ↗
5. Comparison and adaptation of two strategies for anomaly detection in load profiles based on methods from the fields of machine learning and statistics10.4236/ojee.2020.102003 ↗
6. Design and evaluation of outlier detection based on semantic condensed nearest neighbor
7. An efficient automatic gait anomaly detection method based on semisupervised clustering10.1155/2021/8840156 ↗
8. Similarity-measured isolation forest: Anomaly detection method for machine monitoring data
9. Log-based anomaly detection with the improved K-nearest neighbor10.1142/S0218194020500114 ↗
10. Unsupervised anomaly detection based on clustering methods and sensor data on a marine diesel engine10.1080/20464177.2019.1633223 ↗
11. GA-iForest: An efficient isolated forest framework based on genetic algorithm for numerical data outlier detection
12. Lim, S.S., and Kwon, O.W. (2022). Frame augment: A simple data augmentation method for encoder–decoder speech recognition. Appl. Sci., 12.10.3390/app12157619 ↗
13. End-to-end entity-aware neural machine translation10.1007/s10994-021-06073-9 ↗
14. A brief review of new data analysis methods of laser-induced breakdown spectroscopy: Machine learning10.1080/05704928.2020.1843175 ↗
15. Li, Y., Xu, Y., Cao, Y., Hou, J., Wang, C., Guo, W., Li, X., Xin, Y., Liu, Z., and Cui, L. (2022). One-class LSTM network for anomalous network traffic detection. Appl. Sci., 12.10.3390/app12105051 ↗
16. A semisupervised autoencoder-based approach for anomaly detection in high performance computing systems10.1016/j.engappai.2019.07.008 ↗
17. Chen, Z., Yeo, C.K., Lee, B.S., and Lau, C.T. (2018, January 17–20). Autoencoder-based network anomaly detection. Proceedings of the 2018 Wireless Telecommunications Symposium (WTS), Phoenix, AZ, USA.10.1109/WTS.2018.8363930 ↗
18. Liu, F.T., Ting, K.M., and Zhou, Z.H. (2008, January 15–19). Isolation forest. Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, Pisa, Italy.10.1109/ICDM.2008.17 ↗
19. Zhang, W., and Chen, L. (2019, January 14–16). Web log anomaly detection based on isolated forest algorithm. Proceedings of the 2019 IEEE 14th International Conference on Intelligent Systems and Knowledge Engineering (ISKE), Dalian, China.10.1109/ISKE47853.2019.9170430 ↗
20. Mao, W., Cao, X., Zhou, Q., Yan, T., and Zhang, Y. (2018, January 6–8). Anomaly detection for power consumption data based on isolated forest. Proceedings of the 2018 International Conference on Power System Technology (POWERCON), Guangzhou, China.10.1109/POWERCON.2018.8602251 ↗
21. The theoretical breakthrough of self-supervised learning: Variational autoencoders and its application in big data analysis10.1088/1742-6596/1955/1/012062 ↗
22. Novel segmented stacked autoencoder for effective dimensionality reduction and feature extraction in hyperspectral imaging10.1016/j.neucom.2015.11.044 ↗
23. A shallow network for hyperspectral image classification using an autoencoder with convolutional neural network10.1007/s11042-021-11422-w ↗
24. Surface microseismic data denoising based on sparse autoencoder and Kalman filter10.1080/21642583.2022.2087786 ↗
25. Anomaly detection for hydropower turbine unit based on variational modal decomposition and deep autoencoder10.1016/j.egyr.2021.09.179 ↗
26. Improved autoencoder for unsupervised anomaly detection10.1002/int.22582 ↗
27. Seokheon, Y. (2022). Performance analysis of construction cost prediction using neural network for multioutput regression. Appl. Sci., 12.10.3390/app12199592 ↗
28. Contrastive autoencoder for anomaly detection in multivariate time series10.1016/j.ins.2022.07.179 ↗
29. Liang, D., Wang, J., Gao, X., Wang, J., Zhao, X., and Wang, L. (2020, January 20–22). Self-supervised Pretraining Isolated Forest for Outlier Detection. Proceedings of the 2022 International Conference on Big Data, Information and Computer Network (BDICN), Sanya, China.10.1109/BDICN55575.2022.00065 ↗
30. Random search for hyper-parameter optimization