journal article·2024-02-28Open access

Time Series Feature Selection Method Based on Mutual Information

Lin Huang, Xingqiang Zhou, Lianhui Shi, Li Gong

Applied Sciences· Volume 14· Issue 5

Abstract

Time series data have characteristics such as high dimensionality, excessive noise, data imbalance, etc. In the data preprocessing process, feature selection plays an important role in the quantitative analysis of multidimensional time series data. Aiming at the problem of feature selection of multidimensional time series data, a feature selection method for time series based on mutual information (MI) is proposed. One of the difficulties of traditional MI methods is in searching for a suitable target variable. To address this issue, the main innovation of this paper is the hybridization of principal component analysis (PCA) and kernel regression (KR) methods based on MI. Firstly, based on historical operational data, quantifiable system operability is constructed using PCA and KR. The next step is to use the constructed system operability as the target variable for MI analysis to extract the most useful features for the system data analysis. In order to verify the effectiveness of the method, an experiment is conducted on the CMAPSS engine dataset, and the effectiveness of condition recognition is tested based on the extracted features. The results indicate that the proposed method can effectively achieve feature extraction of high-dimensional monitoring data.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Anomaly Detection Techniques and Applications · Time Series Analysis and Forecasting · Advanced Algorithms and Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Ship Comprehensive Test and Training Base, Naval University of Engineering, Wuhan 430033, China

91251 Army of PLA, Shanghai 200940, China

References (49)

Show source-provided references
1. A machine learning approach to circumventing the curse of dimensionality in discontinuous time series machine data10.1016/j.ress.2019.106706 ↗
2. Stability of feature selection algorithm: A review
3. A comprehensive review of dimensionality reduction techniques for feature selection and feature extraction10.38094/jastt1224 ↗
4. Image Recognition Based on Compressive Imaging and Optimal Feature Selection
5. A framework of human action recognition using length control features fusion and weighted entropy-variances based feature selection10.1016/j.imavis.2020.104090 ↗
6. A machine learning-based sentiment analysis of online product reviews with a novel term weighting and feature selection approach10.1016/j.ipm.2021.102656 ↗
7. A Comprehensive Analysis of Nature-Inspired Meta-Heuristic Techniques for Feature Selection Problem10.1007/s11831-020-09412-6 ↗
8. A novel feature selection method for data mining tasks using hybrid Sine Cosine Algorithm and Genetic Algorithm10.1007/s10586-021-03254-y ↗
9. A feature selection method via analysis of relevance, redundancy, and interaction10.1016/j.eswa.2021.115365 ↗
10. Generalized refined composite multiscale fuzzy entropy and multi-cluster feature selection based intelligent fault diagnosis of rolling bearing10.1016/j.isatra.2021.05.042 ↗
11. A Sound-Based Fault Diagnosis Method for Railway Point Machines Based on Two-Stage Feature Selection Strategy and Ensemble Classifier10.1109/TITS.2021.3109632 ↗
12. Bearing fault diagnosis and prognosis using data fusion based feature extraction and feature selection10.1016/j.measurement.2021.110506 ↗
13. Three-stage feature selection approach for deep learning-based RUL prediction methods10.1002/qre.3288 ↗
14. An intelligent feature selection approach with systolic tree structures for efficient association rules in big data environment10.1016/j.compeleceng.2022.108080 ↗
15. Dhindsa, A., Bhatia, S., Agrawal, S., and Sohi, B.S. (2021). An Improvised Machine Learning Model Based on Mutual Information Feature Selection Approach for Microbes Classification. Entropy, 23.10.3390/e23020257 ↗
16. Halomoan, J., Ramli, K., Sudiana, D., Gunawan, T.S., and Salman, M. (2023). ECG-Based Driving Fatigue Detection Using Heart Rate Variability Analysis with Mutual Information. Information, 14.10.3390/info14100539 ↗
17. Islam, M.R., Ahmed, B., Hossain, M.A., and Uddin, M.P. (2023). Mutual Information-Driven Feature Reduction for Hyperspectral Image Classification. Sensors, 23.10.3390/s23020657 ↗
18. A survey on intrusion detection system: Feature selection, model, performance measures, application perspective, challenges, and future research directions10.1007/s10462-021-10037-9 ↗
19. An effective genetic algorithm-based feature selection method for intrusion detection systems10.1016/j.cose.2021.102448 ↗
20. Feature selection for intrusion detection system in Internet-of-Things (IoT)10.1016/j.icte.2021.04.012 ↗
21. Alalhareth, M., and Hong, S. (2023). An Improved Mutual Information Feature Selection Technique for Intrusion Detection Systems in the Internet of Medical Things. Sensors, 23.10.3390/s23104971 ↗
22. A review of recent approaches on wrapper feature selection for intrusion detection10.1016/j.eswa.2022.116822 ↗
23. Feature selection using bare-bones particle swarm optimization with mutual information10.1016/j.patcog.2020.107804 ↗
24. Sánchez-Maroño, N., Alonso-Betanzos, A., and Tombilla-Sanromán, M. (2007). Proceedings of the International Conference on Intelligent Data Engineering and Automated Learning, Birmingham, UK, 16–19 December 2007, Springer.
25. Supervised feature selection techniques in network intrusion detection: A critical review10.1016/j.engappai.2021.104216 ↗
26. Feature selection in machine learning: A new perspective10.1016/j.neucom.2017.11.077 ↗
27. A methodology for energy multivariate time series forecasting in smart buildings based on feature selection10.1016/j.enbuild.2019.05.021 ↗
28. Estimating mutual information10.1103/PhysRevE.69.066138 ↗
29. Theoretical evaluation of feature selection methods based on mutual information10.1016/j.neucom.2016.11.047 ↗
30. Feature selection based on mutual information with correlation coefficient10.1007/s10489-021-02524-x ↗
31. Cheng, J., Sun, J., Yao, K., Xu, M., and Cao, Y. (2022). A variable selection method based on mutual information and variance inflation factor. Spectrochim. Acta Part A Mol. Biomol. Spectrosc., 268.10.1016/j.saa.2021.120652 ↗
32. Feature selection using Fisher score and multilabel neighborhood rough sets for multilabel classification10.1016/j.ins.2021.08.032 ↗
33. A hybrid feature selection method based on information theory and binary butterfly optimization algorithm10.1016/j.engappai.2020.104079 ↗
34. Liang, J., Hou, L., Luan, Z., and Huang, W. (2019). Feature Selection with Conditional Mutual Information Considering Feature Interaction. Symmetry, 11.10.3390/sym11070858 ↗
35. Hammad, M., Chelloug, S.A., Alayed, W., and El-Latif, A.A.A. (2023). Optimizing Multimodal Scene Recognition through Mutual Information-Based Feature Selection in Deep Learning Models. Appl. Sci., 13.10.3390/app132111829 ↗
36. Li, K., and Fard, N. (2022). A Novel Nonparametric Feature Selection Approach Based on Mutual Information Transfer Network. Entropy, 24.10.3390/e24091255 ↗
37. Mutual Information Variational Autoencoders and Its Application to Feature Extraction of Multivariate Time Series10.1142/S0218001422550059 ↗
38. Fuzzy Mutual Information-Based Multilabel Feature Selection with Label Dependency and Streaming Labels10.1109/TFUZZ.2022.3182441 ↗
39. Conditional mutual information-based feature selection algorithm for maximal relevance minimal redundancy10.1007/s10489-021-02412-4 ↗
40. What is principal component analysis?10.1038/nbt0308-303 ↗
41. A Review of Principal Component Analysis Algorithm for Dimensionality Reduction
42. Issues and recommendations for exploratory factor analysis and principal component analysis10.1016/j.sapharm.2020.07.027 ↗
43. Verleysen, M., and François, D. (2005). Proceedings of the International Work-Conference on Artificial Neural Networks, Barcelona, Spain, 8–10 June 2005, Springer.
44. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy10.1109/TPAMI.2005.159 ↗
45. Saxena, A., Goebel, K., Simon, D., and Eklund, N. (2008, January 6–9). Damage propagation modeling for aircraft engine run-to-failure simulation. Proceedings of the 2008 International Conference on Prognostics and Health Management, Denver, CO, USA.10.1109/PHM.2008.4711414 ↗
46. (2024, February 17). Seaborn: Statistical Data Visualization. Available online: https://seaborn.pydata.org/.
47. (2024, February 17). Matplotlib: Visualization with Python. Available online: https://matplotlib.org/.
48. Trajectory Similarity Matching and Remaining Useful Life Prediction Based on Dynamic Time Warping10.1155/2022/5344461 ↗
49. Testing random effects in linear mixed models: Another look at the F-test (with discussion)10.1111/anzs.12256 ↗