A Matrix-Statistics-Aware Attention Mechanism for Robust RUL Estimation in Aero-Engines
Source abstract
Abstract
Prognostics and Health Management (PHM) is a vital approach which aims to predict the failure of engineering systems at an early stage and optimize maintenance strategies. It operates through continuous system monitoring, anomaly detection, fault detection, and Remaining Useful Life (RUL) estimation. Accurate RUL prediction for aircraft engines is critical for enhancing operational safety and minimizing maintenance costs. Traditional methods are largely dependent on handcrafted features and domain-specific knowledge. They often fail to capture the nonlinear and high-dimensional degradation dynamics of real-world systems. In this study, we propose an enhanced deep learning architecture combining Long Short-Term Memory (LSTM) and Bidirectional LSTM networks with a new Matrix-Statistics-Aware Attention (LSTM-MSAA) method. Unlike conventional attention methods, our proposed method incorporates auxiliary scalar features, such as the Frobenius norm, spectral norm, and soft rank, into the attention score computation. This hybrid model provides a more informative representation of engine state transitions. The model is evaluated on both legacy and newly released C-MAPSS datasets from NASA’s Prognostics Data Repository. Experimental results reveal a reduction in RMSE compared to baseline models, validating the effectiveness of our attention fusion strategy in capturing intricate degradation behaviors and improving predictive performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Fault Diagnosis Techniques · Reliability and Maintenance Optimization · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Electrical-Electronic Engineering Department, Bursa Uludag University, Bursa 16059, Türkiye
TUSAS Uludag University R&D Center, Turkish Aerospace Industries, Bursa 16059, Türkiye
Funding
Scientific and Technological Research Council of Turkey
References (81)
Show source-provided references
1. Bridgman, M.S. (2002, January 9–16). Relating failure prognostics to system benefits. Proceedings of the Aerospace Conference, Big Sky, MT, USA.
2. Prognostics and health management (PHM): Where are we and where do we (need to) go in theory and practice10.1016/j.ress.2021.108119 ↗
3. Prognostics and health management design for rotary machinery systems—Reviews, methodology and applications10.1016/j.ymssp.2013.06.004 ↗
4. A review: Prognostics and health management in automotive and aerospace
5. Fu, S., and Avdelidis, N.P. (2023). Prognostic and health management of critical aircraft systems and components: An overview. Sensors, 23.10.3390/s23198124 ↗
6. Prognostic and health management for engineering systems: A review of the data-driven approach and algorithms
7. State of the art and taxonomy of prognostics approaches, trends of prognostics applications and open issues towards maturity at different technology readiness levels10.1016/j.ymssp.2017.01.050 ↗
8. Remaining Useful Life prediction and challenges: A literature review on the use of Machine Learning Methods10.1016/j.jmsy.2022.05.010 ↗
9. Malhotra, P., Vig, L., Shroff, G., and Agarwal, P. (2017, January 26–28). Long short term memory networks for anomaly detection in time series. Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges, Belgium.
10. A review on deep learning applications in prognostics and health management10.1109/ACCESS.2019.2950985 ↗
11. Wang, X., Li, Y., Xu, Y., Liu, X., Zheng, T., and Zheng, B. (2023). Remaining useful life prediction for aero-engines using a time-enhanced multi-head self-attention model. Aerospace, 10.10.3390/aerospace10010080 ↗
12. Remaining useful life prediction using deep learning approaches: A review10.1016/j.promfg.2020.06.015 ↗
13. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is All You Need. arXiv.
14. Bao, P., Yi, W., Zhu, Y., Shen, Y., and Chai, B.X. (2025). STHFD: Spatial–Temporal Hypergraph-Based Model for Aero-Engine Bearing Fault Diagnosis. Aerospace, 12.10.3390/aerospace12070612 ↗
15. NASA (2025, February 17). PCOE Data Set Repository, Available online: https://www.nasa.gov/intelligent-systems-division/discovery-and-systems-health/pcoe/pcoe-data-set-repository/.
16. Frederick, D.K., DeCastro, J.A., and Litt, J.S. (2007). User’s Guide for the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS), NASA Technical Report.10.2514/6.2008-4579 ↗
17. Fan, Z., Li, W., and Chang, K.-C. (2024). A two-stage attention-based hierarchical transformer for turbofan engine remaining useful life prediction. Sensors, 24.10.3390/s24030824 ↗
18. Lab-ver: An LSTM attention based on variational autoencoder representation learning of remaining useful life estimation10.1088/2631-8695/adbe26 ↗
19. Predicting maintenance through an attention long short-term memory projected model10.1007/s10845-023-02077-5 ↗
20. Spatial correlation and temporal attention-based LSTM for remaining useful life prediction of turbofan engine10.1016/j.measurement.2023.112816 ↗
21. Peringal, A., Mohiuddin, M.B., and Hassan, A. (2024). Remaining useful life prediction for aircraft engines using LSTM. arXiv.
22. Self-attention transformer-based architecture for remaining useful life estimation of complex machines10.1016/j.procs.2022.12.241 ↗
23. Zhang, Y., Wang, X., Liu, Y., and Li, H. (2024, January 24–28). Remaining useful life prediction of aircraft engine based on Bi-LSTM network integrated with attention mechanism. Proceedings of the ASME Turbo Expo 2024: Turbomachinery Technical Conference and Exposition, London, UK.
24. Fan, Y., Tang, Q., Guo, Y., and Wei, Y. (2024). BiLSTM-MLAM: A multi-scale time series prediction model for sensor data based on Bi-LSTM and local attention mechanisms. Sensors, 24.10.21203/rs.3.rs-4229520/v1 ↗
25. Yang, W., Zhou, Q., Chen, G., Liu, C., and Qu, C. (2024, January 19–22). SDPA-BiLSTM: Improving accuracy of aircraft engine remaining life prediction with enhanced attention mechanism. Proceedings of the 9th International Conference on Computer and Communication Systems (ICCCS), Xi’an, China.10.1109/ICCCS61882.2024.10603159 ↗
26. Ji, S., Han, X., Hou, Y., Song, Y., and Du, Q. (2020). Remaining useful life prediction of airplane engine based on PCA–BLSTM. Sensors, 20.10.3390/s20164537 ↗
27. Spatial-temporal dual-channel adaptive graph convolutional network for remaining useful life prediction with multi-sensor information fusion10.1016/j.aei.2023.102120 ↗
28. Time-varying trajectory modeling via dynamic governing network for remaining useful life prediction10.1016/j.ymssp.2022.109610 ↗
29. Shaw, P., Uszkoreit, J., and Vaswani, A. (2018). Self-attention with Relative Position Representations. arXiv.10.18653/v1/N18-2074 ↗
30. Wang, S., Li, B.Z., Khabsa, M., Fang, H., and Ma, H. (2020). Linformer: Self-Attention with Linear Complexity. arXiv.
31. Mixformer: An improved self-attention architecture applied to multivariate chaotic time series prediction10.1016/j.eswa.2023.122484 ↗
32. Primal-attention: Self-attention through asymmetric kernel SVD in primal representation
33. Aranda, J. (2024). Singular value decomposition (SVD) as a key to efficient attention mechanisms and hierarchical structure learning. ResearchGate.
34. A novel local enhanced channel self-attention based on Transformer for industrial remaining useful life prediction10.1016/j.engappai.2024.109815 ↗
35. An adaptive dual distillation framework for efficient remaining useful life prediction10.1007/s40747-025-01886-w ↗
36. Enhanced Remaining Useful Life Prediction for Turbofan Engines Using Spatio-Temporal Koopman Dual-branch Transformer10.1109/TIM.2025.3625335 ↗
37. Blondel, M., Teboul, O., Berthet, Q., and Djolonga, J. (2020, January 12–18). Fast differentiable sorting and ranking. Proceedings of the 37th International Conference on Machine Learning, Vienna, Austria.
38. Constraint-guided learning of data-driven health indicator models: An application on bearings
39. Arias Chao, M., Kulkarni, C., Goebel, K., and Fink, O. (2021). Aircraft engine run-to-failure dataset under real flight conditions for prognostics and diagnostics. Data, 6.10.3390/data6010005 ↗
40. Vollert, S., and Theissler, A. (2021, January 7–10). Challenges of machine learning-based RUL prognosis: A review on NASA’s C-MAPSS data set. Proceedings of the 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Vasteras, Sweden.10.1109/ETFA45728.2021.9613682 ↗
41. Saxena, A., Goebel, K., Simon, D., and Eklund, N. (2008, January 6–9). Damage propagation modeling for aircraft engine run-to-failure simulation. Proceedings of the International Conference on Prognostics and Health Management, Denver, CO, USA.10.1109/PHM.2008.4711414 ↗
42. Zhang, A., Wang, H., Li, S., Cui, Y., Liu, Z., Yang, G., and Hu, J. (2018). Transfer learning with deep recurrent neural networks for remaining useful life estimation. Appl. Sci., 8.10.3390/app8122416 ↗
43. Performance benchmarking and analysis of prognostic methods for C-MAPSS datasets
44. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
45. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Chapter 10.
46. Enhanced fault classification in bearings: A multi-domain feature extraction approach with LSTM-attention and LASSO
47. Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q.V., and Salakhutdinov, R. (2019). Transformer-XL: Attentive language models beyond a fixed-length context. arXiv.10.18653/v1/P19-1285 ↗
48. Nonlinear hysteretic parameter identification using an attention-based long short-term memory network and principal component analysis10.1007/s11071-022-08095-x ↗
49. Thoiyab, N.M., Shanmugam, S., Vadivel, R., and Gunasekaran, N. (2025). Frobenius norm-based global stability analysis of delayed bidirectional associative memory neural networks. Symmetry, 17.10.3390/sym17020183 ↗
50. Ford, W. (2014). Numerical Linear Algebra with Applications: Using MATLAB and Octave, Academic Press.
51. Combating spatial redundancy with spectral norm attention in convolutional learners10.1016/j.neucom.2022.09.075 ↗
52. Golub, G.H., and Van Loan, C.F. (2013). Matrix Computations, Johns Hopkins University Press. [4th ed.].
53. Mohr, D.L., Wilson, W.J., and Freund, R.J. (2021). Statistical Methods, Academic Press. [4th ed.].
54. Huang, Y., Tao, J., Sun, G., Zhang, H., and Hu, Y. (2022). A prognostic and health management framework for aero-engines based on a dynamic probability model and LSTM network. Aerospace, 9.10.3390/aerospace9060316 ↗
55. Aero-engine prognosis strategy based on multi-scale feature fusion and multi-task parallel learning10.1016/j.ress.2023.109182 ↗
56. Narwariya, J., Malhotra, P., TV, V., Vig, L., and Shroff, G. (2020). Graph neural networks for leveraging industrial equipment structure: An application to remaining useful life estimation. arXiv.
57. Remaining useful life estimation via transformer encoder enhanced by a gated convolutional unit10.1007/s10845-021-01750-x ↗
58. Machine remaining useful life prediction via an attention-based deep learning approach10.1109/TIE.2020.2972443 ↗
59. Fusing physics-based and deep learning models for prognostics10.1016/j.ress.2021.107961 ↗
60. Evolutionary optimization of convolutional extreme learning machine for remaining useful life prediction10.1007/s42979-023-02320-z ↗
61. Wen, Z., Fang, Y., Wei, P., Liu, F., Chen, Z., and Wu, M. (2024). Temporal and heterogeneous graph neural network for remaining useful life prediction. arXiv.10.1109/TNNLS.2025.3592788 ↗
62. Collaborative training of data-driven remaining useful life prediction models using federated learning
63. Enhancing non-stationary feature learning for remaining useful life prediction of aero-engine under multiple operating conditions10.1016/j.measurement.2024.114242 ↗
64. Remaining useful life prediction based on graph feature attention networks with missing multi-sensor features10.1016/j.ress.2025.110902 ↗
65. Self-supervised learning for remaining useful life prediction using simple triplet networks10.1016/j.aei.2024.103038 ↗
66. A hybrid prognosis method based on health indicator and Wiener process: The case of multi-sensor monitored aero-engine10.1016/j.engappai.2025.110099 ↗
67. Robust uncertainty quantification for online remaining useful life prediction with randomly missing and partially faulty sensor data10.1016/j.ress.2025.111177 ↗
68. KDMFusion: Knowledge-guided dual-path multi-feature fusion framework for remaining useful life prediction10.1109/JSEN.2025.3547323 ↗
69. Operating condition feature representation-based Fourier graph network for civil aircraft state estimation10.1016/j.ress.2025.111085 ↗
70. Cui, H., Guo, X., and Yu, L. (2025). Leveraging Pre-Trained GPT Models for Equipment Remaining Useful Life Prognostics. Electronics, 14.10.3390/electronics14071265 ↗
71. Zhu, Y., and Liu, Z. (2024). A dual-dimension convolutional-attention module for remaining useful life prediction of aeroengines. Aerospace, 11.10.3390/aerospace11100809 ↗
72. A predictive maintenance strategy using deep learning quantile regression and kernel density estimation for failure prediction
73. Uncertainty-aware prognosis via deep Gaussian process10.1109/ACCESS.2021.3110049 ↗
74. Berghout, T., Mouss, M.D., Mouss, L.H., and Benbouzid, M. (2022). ProgNet: A transferable deep network for aircraft engine damage propagation prognosis under real flight conditions. Aerospace, 10.10.3390/aerospace10010010 ↗
75. Mo, H., and Iacca, G. (2022, January 20–22). Multi-objective optimization of extreme learning machine for remaining useful life prediction. Proceedings of the International Conference on Applications of Evolutionary Computation (Part of EvoStar), Madrid, Spain.10.1007/978-3-031-02462-7_13 ↗
76. Spatio-temporal degradation modeling and remaining useful life prediction under multiple operating conditions based on attention mechanism and deep learning10.1016/j.ress.2022.108886 ↗
77. DLformer: A dynamic length transformer-based network for efficient feature representation in remaining useful life prediction10.1109/TNNLS.2023.3257038 ↗
78. Comprehensive dynamic structure graph neural network for aero-engine remaining useful life prediction10.1109/TIM.2023.3322481 ↗
79. Multi-resolution LSTM-based prediction model for remaining useful life of aero-engine10.1109/TVT.2023.3319377 ↗
80. A benchmark on uncertainty quantification for deep learning prognostics10.1016/j.ress.2024.110513 ↗
81. A core space gradient projection-based continual learning framework for remaining useful life prediction of machinery under variable operating conditions10.1016/j.ress.2024.110428 ↗