Generative Artificial Intelligence and Machine Translators in Spanish Translation of Early Vulnerability Cybersecurity Alerts
Source abstract
Abstract
The increasing reliance on artificial intelligence in cybersecurity has broadened the role of generative artificial intelligence in tasks such as text generation and translation. This study assesses the effectiveness of generative artificial intelligence and conventional translation tools in translating early vulnerability alerts from English to Spanish—a critical process for ensuring the timely dissemination of cybersecurity information. Utilizing a dataset provided by the Spanish National Cybersecurity Institute, translations were generated using various systems and evaluated through linguistic assessment metrics, including methods measuring lexical similarity and others capturing semantic meaning beyond direct word matching. Additionally, word embeddings were employed to enhance the accuracy of semantic similarity analysis. The results indicate that conventional translation tools generally exhibit greater accuracy and structural fidelity, whereas generative artificial intelligence produces more natural-sounding translations. However, this flexibility results in greater variability in translation quality. The findings suggest that while generative artificial intelligence may serve as a valuable complement to traditional tools, its inconsistencies may limit its suitability for highly technical content that demands precision. This study underscores the importance of integrating both approaches to improve the accuracy and accessibility of cybersecurity alerts across different languages.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Misinformation and Its Impacts
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Research Institute of Applied Sciences in Cybersecurity (RIASC), Universidad de León, Campus de Vegazana, s/n, 24007 León, Spain
Department of Mathematics, Universidad de León, Campus de Vegazana, s/n, 24007 León, Spain
Funding
Strategic Project Data Science for an Artificial Intelligence Model in Cybersecurity · European Union
References (64)
Show source-provided references
1. NIST (2024, November 11). Generative Artificial Intelligence, Available online: https://csrc.nist.gov/glossary/term/generative_artificial_intelligence.
2. Sengar, S.S., Hasan, A.B., Kumar, S., and Carroll, F. (2024). Generative artificial intelligence: A systematic review and applications. Multimed. Tools Appl.10.1007/s11042-024-20016-1 ↗
3. A Comparative Analysis of Generative Artificial Intelligence Responses from Leading Chatbots to Questions About Endometriosis10.1016/j.xagr.2024.100405 ↗
4. NIST (2025, January 15). National Institute of Standards and Technology, Available online: https://www.nist.gov/.
5. INCIBE (2025, February 28). INCIBE Vulnerabilities. Available online: https://www.incibe.es/incibe-cert/alerta-temprana/vulnerabilidades.
6. Artificial intelligence and posthumanist translation: ChatGPT versus the translator10.1515/applirev-2023-0122 ↗
7. A Comparative Analysis of Generative Artificial Intelligence Tools for Natural Language Processing10.62411/jcta.9447 ↗
8. INCIBE (2025, February 28). Vulnerability Database. Available online: https://drive.google.com/file/d/1tJOP8R98DcEMCNh6whXkqTsXGFpoTYU2/view?usp=sharing.
9. National Institute of Standards and Technology (2012). Risk Management Guide for Information Technology Systems, National Institute of Standards and Technology. Technical Report NIST SP 800-30 Rev. 1.
10. IBM (2025, February 28). Cost of a Data Breach Report 2024. Available online: https://www.ibm.com/reports/data-breach.
11. MITRE Corporation (2025, February 05). Common Vulnerabilities and Exposures (CVE)—Introduction. Available online: https://cve.mitre.org/docs/cve-intro-handout.pdf.
12. Farrell, M. (2024, January 18–20). Survey on the Use of Generative Artificial Intelligence by Professional Translators. Proceedings of the 46th Conference Translating and the Computer, Luxembourg.
13. What are the differences? A comparative study of generative artificial intelligence translation and human translation of scientific texts10.1057/s41599-024-03726-7 ↗
14. EAMT (2025, January 16). European Association for Machine Translation. Available online: https://eamt.org.
15. Hutchins, W.J. (1986). Machine Translation: Past, Present, Future, Academic Press.
16. Koehn, P. (2003). Statistical Machine Translation, Cambridge University Press.10.21236/ADA461156 ↗
17. Bahdanau, D., Cho, K., and Bengio, Y. (2015, January 7–9). Neural Machine Translation by Jointly Learning to Align and Translate. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA.
18. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is All You Need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
19. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv.
20. DeepL SE (2025, February 28). DeepL. Available online: https://www.deepl.com/es/translator.
21. Google (2025, February 28). Google Translate. Available online: https://translate.google.es/?hl=es&sl=fr&tl=es&op=translate.
22. Microsoft (2025, February 28). Bing Translator. Available online: https://www.bing.com/translator?to=es&setlang=es.
23. Chapsvision (2025, February 17). Systran. Available online: https://www.systransoft.com/es/.
24. Softissimo (2025, February 17). Reverso. Available online: https://www.reverso.net/traducci%C3%B3n-texto.
25. OpenAI (2025, February 28). ChatGPT. Available online: https://openai.com/chatgpt.
26. Google (2025, February 28). Google Bard (Gemini). Available online: https://gemini.google.com/?hl=es-ES.
27. Microsoft (2025, January 05). Microsoft Copilot. Available online: https://copilot.microsoft.com/.
28. Google (2025, January 05). Google Trends. Available online: https://trends.google.es/trends/.
29. (2025, February 28). Comisión Europea, ETranslation Info. Available online: https://language-tools.ec.europa.eu/documentation.
30. (2025, February 28). Comisión Europea, ETranslation API. Available online: https://webgate.ec.europa.eu/etranslation/public/requestApiKey.html.
31. Pangeanic (2025, February 17). Pangeanic. Available online: https://pangeanic.com/es/.
32. Chapsvision (2025, February 28). Systran API. Available online: https://pangeanic.com/translation-technology/translate-easy.
33. Softissimo (2025, February 28). Reverso API. Available online: https://www.npmjs.com/package/reverso-api.
34. (2025, February 17). Ministerio para la Transformación digital y de la función pública, Gobierno de España, ALIA—Artificial Inteligence. Available online: https://www.alia.gob.es/.
35. Nidhaloff (2025, February 28). DeepTranslator GitHub. Available online: https://github.com/nidhaloff/deep-translator.
36. Klein, G., Kim, Y., Deng, Y., Senellart, J., and Rush, A. (2025, February 28). OpenNMT. Available online: https://opennmt.net/.
37. vince62s (2025, February 17). Eole-nlp. Available online: https://github.com/eole-nlp/eole?tab=readme-ov-file.
38. Natural Language Processing Group T.U. (2025, April 03). EOLE. Available online: https://github.com/eole-nlp/eole.
39. DeepSeek (2025, January 31). DeepSeek. Available online: https://www.deepseek.com/.
40. X (2025, January 08). Grok. Available online: https://x.ai/.
41. Amazon (2025, January 08). Amazon Nova. Available online: https://aws.amazon.com/es/ai/generative-ai/nova/.
42. Gough, D., Oliver, S., and Thomas, J. (2012). An Introduction to Systematic Reviews, SAGE.
43. Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 7–12). BLEU: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Philadelphia, PA, USA.10.3115/1073083.1073135 ↗
44. Wu, Y., Schuster, M., Chen, Z., Le, Q.V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., and Macherey, K. (2016). Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv.
45. Popovic, M. (2015, January 17–18). chrF: Character n-gram F-score for automatic MT evaluation. Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisbon, Portugal.10.18653/v1/W15-3049 ↗
46. Banerjee, S., and Lavie, A. (2005, January 29). METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, MI, USA.
47. Isozaki, H., Hirao, T., Duh, K., Sudoh, K., and Tsukada, H. (2010, January 9–11). Automatic Evaluation of Translation Quality for Distant Language Pairs. Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Cambridge, MA, USA.
48. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. (2006, January 8–12). A Study of Translation Edit Rate with Targeted Human Annotation. Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, Cambridge, MA, USA.
49. Lo, C.-k. (2019, January 1–2). YiSi—A Unified Semantic MT Quality Evaluation and Estimation Metric for Languages with Different Levels of Available Resources. Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), Florence, Italy.10.18653/v1/W19-5358 ↗
50. Rei, R., Stewart, C., Farinha, A.C., and Lavie, A. (2020, January 16–20). COMET: A Neural Framework for MT Evaluation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.10.18653/v1/2020.emnlp-main.213 ↗
51. Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., and Artzi, Y. (2019). BERTScore: Evaluating Text Generation with BERT. arXiv.
52. Gupta, P., Shekhawat, S., and Kumar, K. (February, January 30). Unsupervised Quality Estimation Without Reference Corpus for Subtitle Machine Translation Using Word Embeddings. Proceedings of the 2019 IEEE 13th International Conference on Semantic Computing (ICSC), Newport Beach, CA, USA.10.1109/ICOSC.2019.8665529 ↗
53. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013, January 2–4). Efficient Estimation of Word Representations in Vector Space. Proceedings of the Workshop at ICLR, Scottsdale, AZ, USA.
54. Pennington, J., Socher, R., and Manning, C.D. (2014, January 25–29). GloVe: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1162 ↗
55. Enriching Word Vectors with Subword Information10.1162/tacl_a_00051 ↗
56. A generalized solution of the orthogonal procrustes problem10.1007/BF02289451 ↗
57. INCIBE (2024, November 26). Descriptions of Vulnerabilities in the NVD Database. Available online: https://elrc-share.eu/repository/browse/descripciones-de-vulnerabilidades-de-la-bbdd-nvd/9dd01c02cfcc11e9913100155d02670670070560807f458f8e50bd5273778d7a/.
58. Conneau, A., Lample, G., Ranzato, M.A., Denoyer, L., and Jégou, H. (2017). Word Translation Without Parallel Data. arXiv, Available online: https://github.com/facebookresearch/MUSE.
59. Developers, S. (2024, November 26). SciPy: Open Source Scientific Tools for Python. Available online: https://www.scipy.org/.
60. Bird, S., and Loper, E. (2024, November 11). Natural Language Toolkit (NLTK). Available online: https://github.com/nltk/nltk.
61. chikiulo (2024, September 26). YiSi. Available online: https://github.com/chikiulo/yisi.
62. Lightning AI (2024, November 11). Torchmetrics: Machine Learning Metrics for PyTorch. Available online: https://lightning.ai/docs/torchmetrics/stable/.
63. nttcslab nlp (2024, November 18). RIBES. Available online: https://github.com/nttcslab-nlp/RIBES.
64. Unbabel (2024, September 26). COMET. Available online: https://huggingface.co/Unbabel/wmt20-comet-da.