journal article·2025-08-27Open access

Artificial Intelligence and Aviation: A Deep Learning Strategy for Improved Data Classification and Management

Flávio L. Lázaro, Luís F. F. M. Santos, Duarte Valério, Rui Melicio

Applied Sciences· Volume 15· Issue 17

Abstract

Deep learning (DL) and machine learning (ML) models have been successfully applied across multiple domains, but generic architectures often underperform without domain-specific adaptation. This study presents A-BERT, a BERT-based model fine-tuned on a dataset of aviation and aircraft-related academic publications, enabling accurate classification into 14 thematic categories. The temporal evolution of publication counts in each category was then modeled using ARIMA to forecast future research trends in the aviation sector. As a proof of concept, A-BERT outperformed the baseline BERT in several key metrics, offering a reliable approach for large-scale, domain-specific literature classification. Forecast validation through walk-forward testing across multiple time windows yielded Root Mean Square Error (RMSE) values below 2% for all categories, confirming high predictive reliability within this controlled setting. While the framework demonstrates the potential of combining domain-specific text classification with validated time series forecasting, its extension to operational aviation datasets will require further adaptation and external validation.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Forecasting Techniques and Applications · Time Series Analysis and Forecasting · Air Quality Monitoring and Forecasting

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Institute of Mechanical Engineering (IDMEC), Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal

Faculdade de Engenharia, Universidade Agostinho Neto, Av. 21 de Janeiro, Luanda 1756, Angola

ISEC Lisboa, Alameda das Linhas de Torres, 179, 1750-142 Lisboa, Portugal

Aeronautics and Astronautics Research Center (AEROG), Universidade da Beira Interior, Calçada Fonte do Lameiro, 6200-358 Covilhã, Portugal

Synopsis Planet, Advance Engineering Unipessoal LDA, Faculdade de Ciências, Universidade de Lisboa, Campo Grande 16, 1749-016 Lisboa, Portugal

Funding

MESCTI · Laboratório Associado em Energia, Transportes e Aeroespacial (LAETA) Base Funding · LAETA Programmatic Funding

References (61)

Show source-provided references
1. Applying systems modeling language in an aviation maintenance system10.1109/TEM.2021.3089438 ↗
2. Madeira, T., Melicio, R., Valério, D., and Santos, L. (2021). Machine learning and natural language processing for prediction of human factors in aviation incident reports. Aerospace, 8.10.3390/aerospace8020047 ↗
3. Keller, R.M. (2016, January 25–29). Ontologies for aviation data management. Proceedings of the Digital Avionics Systems Conference (DASC), Sacramento, CA, USA.
4. Lázaro, F.L., Nogueira, R.P.R., Melicio, R., Valério, D., and Santos, L.F.F.M. (2024). Human Factors as Predictor of Fatalities in Aviation Accidents: A Neural Network Analysis. Appl. Sci., 14.10.3390/app14020640 ↗
5. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). BERT: Pre training of deep bidirectional transformers for language understanding. arXiv.
6. Samarra, J., Santos, L.F., Barqueira, A., Melicio, R., and Valério, D. (2023). Uncovering the hidden correlations between socio-economic indicators and aviation accidents in the United States. Appl. Sci., 13.10.3390/app13147997 ↗
7. Probabilistic and statistical analysis of aviation accidents
8. Andrade, S.R., and Walsh, H.S. (2023). SafeAeroBERT: Towards a Safety-Informed Aerospace-Specific Language Model. AIAA AVIATION 2023 Forum, American Institute of Aeronautics and Astronautics (AIAA). Paper AIAA 2023 3437.10.2514/6.2023-3437 ↗
9. Tikayat Ray, A., Cole, B.F., Pinon Fischer, O.J., White, R.T., and Mavris, D.N. (2023). aeroBERT-Classifier: Classification of Aerospace Requirements Using BERT. Aerospace, 10.10.20944/preprints202302.0077.v1 ↗
10. New, M.D., and Wallace, R.J. (2025). Classifying Aviation Safety Reports: Using Supervised Natural Language Processing (NLP) in an Applied Context. Safety, 11.10.3390/safety11010007 ↗
11. Beltagy, I., Lo, K., and Cohan, A. (2019). SciBERT: A Pretrained Language Model for Scientific Text. arXiv.10.18653/v1/D19-1371 ↗
12. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv.
13. Cholectriplet2021: A benchmark challenge for surgical action triplet recognition
14. MFC GAN: Class imbalanced dataset classification using multiple fake class generative adversarial network10.1016/j.neucom.2019.06.043 ↗
15. Hashemi, A., and Dowlatshahi, M. (2023). Neural Networks and Deep Learning. Neural Networks and Deep Learning, Springer Nature. Chapter 1.10.1007/978-981-19-8851-6_13-1 ↗
16. Deep learning neural networks in fuzzy modeling
17. Zhang, C. (May, January 29). Text classification using deep learning methods. Proceedings of the 2022 Conference on Topics in Computing Systems, New Orleans, LA USA.10.1109/TOCS56154.2022.10015956 ↗
18. Liang, K., and Sakakibara, Y. (2021). MetaVelvet DL: A MetaVelvet deep learning extension for de novo metagenome assembly. BMC Bioinform., 22.10.1186/s12859-020-03737-6 ↗
19. Sahu, S.K., Mokhade, A., and Bokde, N.D. (2023). An overview of machine learning, deep learning, and reinforcement learning based techniques in quantitative finance: Recent progress and challenges. Appl. Sci., 13.10.3390/app13031956 ↗
20. Kouris, P., Alexandridis, G., and Stafylopatis, A. (2022). Text summarization based on semantic graphs: An abstract meaning representation graph to text deep learning approach. Res. Sq., preprint.10.21203/rs.3.rs-1938526/v1 ↗
21. An idea based on sequential pattern mining and deep learning for text summarization
22. Gasparetto, A., Marcuzzo, M., Zangari, A., and Albarelli, A. (2022). A survey on text classification algorithms: From text to predictions. Information, 13.10.3390/info13020083 ↗
23. Bidirectional encoder representations from transformers in radiology: A systematic review of natural language processing applications10.1016/j.jacr.2024.01.012 ↗
24. Automated detection of contractual risk clauses from construction specifications using bidirectional encoder representations from transformers (BERT)10.1016/j.autcon.2022.104465 ↗
25. Bidirectional encoder representations from transformers for modelling stock prices10.22214/ijraset.2022.40406 ↗
26. Advancing natural language processing (NLP) applications of morphologically rich languages with bidirectional encoder representations from transformers (BERT): An empirical case study for Turkish
27. Özdil, U., Arslan, B., Taşar, D.E., Polat, G., and Ozan, Ş. (2021, January 15–17). Ad text classification with bidirectional encoder representations. Proceedings of the 2021 6th International Conference on Computer Science and Engineering (UBMK), Ankara, Turkey.10.1109/UBMK52708.2021.9558966 ↗
28. Nanyonga, A., Wasswa, H., Joiner, K., Turhan, U., and Wild, G. (2025). A Multi-Head Attention-Based Transformer Model for Predicting Causes in Aviation Incidents. Modelling, 6.10.20944/preprints202502.1196.v1 ↗
29. Liu, H., Shen, F., Qin, H., and Gao, F. (2024). Research on Flight Accidents Prediction Based on Back Propagation Neural Network. arXiv.10.2991/978-94-6463-514-0_65 ↗
30. Optimization of Thermal-Fluid-Structure Coupling for Variable-Span Inflatable Wings Considering Case Correlation10.1016/j.ast.2024.109448 ↗
31. Generic Deep-Learning-Based Time Series Models for Aviation Accident Analysis and Forecasting
32. Civil aviation satellite navigation integrity monitoring with deep learning10.26855/acc.2023.08.008 ↗
33. Nogueira, R., Melicio, R., Valério, D., and Santos, L. (2023). Learning methods and predictive modeling to identify failure by human factors in the aviation industry. Appl. Sci., 13.10.3390/app13064069 ↗
34. Sequential deep learning from NTSB reports for aviation safety prognosis10.1016/j.ssci.2021.105390 ↗
35. Deep learning based foreign object detection method for aviation runways
36. Toward automated instructor pilots in legacy air force systems: Physiology based flight difficulty classification via machine learning10.1016/j.eswa.2023.120711 ↗
37. Machine learning and mixed reality for smart aviation: Applications and challenges10.1016/j.jairtraman.2023.102437 ↗
38. Automatic Perception of Aircraft Taxiing Behavior via Laser Rangefinders and Machine Learning10.1109/JSEN.2024.3510568 ↗
39. Liang, Z., Zhao, Y., Wang, M., Huang, H., and Xu, H. (2025). Research on the Automatic Multi-Label Classification of Flight Instructor Comments Based on Transformer and Graph Neural Networks. Aerospace, 12.10.3390/aerospace12050407 ↗
40. Human-Factors-in-Aviation-Loop: Multimodal Deep Learning for Pilot Situation Awareness Analysis Using Gaze Position and Flight Control Data10.1109/TITS.2025.3558085 ↗
41. Deep learning and machine learning algorithms for enhanced aircraft maintenance and flight data analysis10.53759/9852/JRS202301009 ↗
42. Lázaro, F.L., Madeira, T., Melicio, R., Valério, D., and Santos, L.F.F.M. (2025). Identifying human factors in aviation accidents with natural language processing and machine learning models. Aerospace, 12.10.3390/aerospace12020106 ↗
43. A multi-objective fuzzy optimization model for multi-type aircraft flight scheduling problem10.3846/transport.2024.20536 ↗
44. Yang, C., and Huang, C. (2023). Natural Language Processing (NLP) in Aviation Safety: Systematic Review of Research and Outlook into the Future. Aerospace, 10.10.3390/aerospace10070600 ↗
45. Fredriksson, T., Bosch, J., and Olsson, H.H. (2020, January 7–9). Machine learning models for automatic labeling: A systematic literature review. Proceedings of the 15th International Conference on Software Technologies (ICSOFT), Paris, France.10.5220/0009972705520561 ↗
46. Forecasting inflation: Autoregressive integrated moving average model
47. Zou, Y., Wang, T., Xiao, J., and Feng, X. (2017, January 19–21). Temperature prediction of electrical equipment based on autoregressive integrated moving average model. Proceedings of the 2017 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC), Hefei, China.10.1109/YAC.2017.7967404 ↗
48. Yang, Y., Wu, W., and Sun, L. (2017, January 14–16). Prediction of mechanical equipment vibration trend using autoregressive integrated moving average model. Proceedings of the 10th International Congress on Image and Signal Processing, Biomedical Engineering and Informatics (CISP-BMEI), Shanghai, China.10.1109/CISP-BMEI.2017.8302110 ↗
49. Seasonal autoregressive integrated moving average for climate change time series forecasting
50. Chodakowska, E., Nazarko, J., and Nazarko, Ł. (2021). ARIMA Models in Electrical Load Forecasting and Their Robustness to Noise. Energies, 14.10.3390/en14237952 ↗
51. Yuwei, C., and Kaizhi, W. (2019, January 19–21). Prediction of satellite time series data based on long short term memory–autoregressive integrated moving average model (LSTM-ARIMA). Proceedings of the 2019 IEEE 4th International Conference on Signal and Image Processing (ICSIP), Wuxi, China.
52. The comparison in time series forecasting of air traffic data by ARIMA, radial basis function and Elman recurrent neural networks
53. Autoregressive integrated moving average (ARIMA) models for birth forecasting10.1080/01621459.1977.10480989 ↗
54. Hybridization of autoregressive integrated moving average (ARIMA) with probabilistic neural networks (PNNs)10.1016/j.cie.2012.01.017 ↗
55. Subhash, N.N., and Minakshee, P.M. (2015, January 21–22). Forecasting telecommunications data with ARIMA models. Proceedings of the 2015 International Conference on Recent Advances in Engineering & Computational Sciences (RAECS), Chandigarh, India.
56. He, P., and Sun, R. (2023). Trend Analysis of Civil Aviation Incidents Based on Causal Inference and Statistical Inference. Aerospace, 10.10.3390/aerospace10090822 ↗
57. Schneider, P., and Xhafa, F. (2022). Anomaly Detection: Concepts and Methods. Anomaly Detection and Complex Event Processing over IoT Data Streams, Academic Press.10.1016/B978-0-12-823818-9.00013-4 ↗
58. A Modified Mann–Kendall Trend Test for Autocorrelated Data10.1016/S0022-1694(97)00125-X ↗
59. Towards the Automated Evaluation of Legal Casenote Essays
60. Oliveira, J.M., and Ramos, P. (2024). Evaluating the Effectiveness of Time Series Transformers for Demand Forecasting in Retail. Mathematics, 12.10.3390/math12172728 ↗
61. Kontopoulou, V.I., Panagopoulos, A.D., Kakkos, I., and Matsopoulos, G.K. (2023). A Review of ARIMA vs. Machine Learning Approaches for Time Series Forecasting in Data Driven Networks. Future Internet, 15.10.3390/fi15080255 ↗