PROMPT-BART: A Named Entity Recognition Model Applied to Cyber Threat Intelligence
Source abstract
Abstract
The growing sophistication of cyberattacks underscores the need for the automated extraction of machine-readable intelligence from unstructured Cyber Threat Intelligence (CTI), commonly achieved through Named Entity Recognition (NER). However, existing CTI-oriented NER research faces two major limitations: the scarcity of standardized datasets and the lack of advanced models tailored to domain-specific entities. To address the dataset challenge, we present CTINER, the first STIX 2.1-aligned dataset, comprising 42,549 annotated entities across 13 cybersecurity-specific types. CTINER surpasses existing resources in both scale (+51.82% more annotated entities) and vocabulary coverage (+40.39%), while ensuring label consistency and rationality. To tackle the modeling challenge, we propose PROMPT-BART, a novel NER model built upon the BART generative framework and enhanced through three types of prompt designs. Experimental results show that PROMPT-BART improves F1 scores by 4.26–8.3% over conventional deep learning baselines and outperforms prompt-based baselines by 1.31%.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Spam and Phishing Detection · Web Data Mining and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Information Engineering College, Capital Normal University, Beijing 100048, China
School of Computing and Information, University of Pittsburgh, Pittsburgh, PA 15260, USA
School of Cyberspace Security, Beijing Institute of Technology, Beijing 100081, China
Funding
Open Foundation of the Key Laboratory of Cyberspace Security, Ministry of Education of China
References (49)
Show source-provided references
1. European Union Agency for Cybersecurity (ENISA) (2024). ENISA Threat Landscape 2024, European Union Agency for Cybersecurity (ENISA). Twelve-Edition Annual Cybersecurity Threat Landscape Report, Covering July 2023–June 2024.
2. Combating Advanced Persistent Threats: Challenges and Solutions10.1109/MNET.2024.3389734 ↗
3. Moskal, S., and Yang, S.J. (2020). Cyberattack Action-Intent-Framework for Mapping Intrusion Observables. arXiv, Available online: https://arxiv.org/abs/2002.07838.
4. A Comprehensive Survey on Advanced Persistent Threat (APT) Detection Techniques
5. Rahman, M.R., Basak, S.K., Hezaveh, R.M., and Williams, L. (2024). Attackers reveal their arsenal: An investigation of adversarial techniques in CTI reports. arXiv, Available online: https://arxiv.org/abs/2401.01865.
6. Cyber-threat intelligence for security decision-making: A review and research agenda for practice10.1016/j.cose.2023.103352 ↗
7. Kulikova, O., Heil, R., van den Berg, J., and Pieters, W. (2012, January 14–16). Cyber Crisis Management: A Decision-Support Framework for Disclosing Security Incident Information. Proceedings of the 2012 International Conference on Cyber Security, Alexandria, VA, USA.10.1109/CyberSecurity.2012.20 ↗
8. Ahrend, J.M., Jirotka, M., and Jones, K. (2016, January 13–14). On the collaborative practices of cyber threat intelligence analysts to develop and utilize tacit Threat and Defence Knowledge. Proceedings of the 2016 International Conference On Cyber Situational Awareness, Data Analytics And Assessment (CyberSA), London, UK.10.1109/CyberSA.2016.7503279 ↗
9. TriCTI: An actionable cyber threat intelligence discovery system via trigger-enhanced neural network10.1186/s42400-022-00110-3 ↗
10. Survey of attack and detection based on the full life cycle of APT
11. Schröer, S.L., Canevascini, N., Pekaric, I., Widmer, P., and Laskov, P. (2025). The Dark Side of the Web: Towards Understanding Various Data Sources in Cyber Threat Intelligence. arXiv, Available online: https://arxiv.org/abs/2504.14235.10.1109/EuroSPW67616.2025.00015 ↗
12. Venčkauskas, A., Toldinas, J., Morkevičius, N., and Sanfilippo, F. (2024). An Email Cyber Threat Intelligence Method Using Domain Ontology and Machine Learning. Electronics, 13.10.3390/electronics13142716 ↗
13. Evangelatos, P., Iliou, C., Mavropoulos, T., Apostolou, K., Tsikrika, T., Vrochidis, S., and Kompatsiaris, I. (2021, January 26–28). Named Entity Recognition in Cyber Threat Intelligence Using Transformer-based Models. Proceedings of the 2021 IEEE International Conference on Cyber Security and Resilience (CSR), Rhodes, Greece.10.1109/CSR51186.2021.9527981 ↗
14. Chinese Cyber Threat Intelligence Named Entity Recognition via RoBERTa-wwm-RDCNN-CRF
15. A Survey of Cyber Threat Intelligence Entity Recognition Research
16. OASIS Cyber Threat Intelligence (CTI) Technical Committee (2025, September 03). STIX 2.1. OASIS, 2019. STIX Version 2.1 Specification. Available online: https://docs.oasis-open.org/cti/stix/v2.1/csprd01/stix-v2.1-csprd01.html.
17. Ma, X., and Hovy, E. (2016, January 7–12). End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.10.18653/v1/P16-1101 ↗
18. Pinheiro, P., and Collobert, R. (2014, January 21–26). Recurrent convolutional neural networks for scene labeling. Proceedings of the 31st International Conference on Machine Learning, Beijing, China.
19. Jiang, S., Zhao, S., and Hou, K. (2019, January 26–27). A BERT-BiLSTM-CRF model for Chinese electronic medical records named entity recognition. Proceedings of the 12th International Conference on Intelligent Computation Technology and Automation (ICICTA), Xiangtan, China.
20. Fang, J., Wang, X., and Meng, Z. (2023, January 9–14). MANNER: A variational memory-augmented model for cross domain few-shot named entity recognition. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, ON, Canada.10.18653/v1/2023.acl-long.234 ↗
21. Chen, W., Zhao, L., and Luo, P. (2023, January 21–25). Heproto: A hierarchical enhancing protonet based on multi-task learning for few-shot named entity recognition. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, Birmingham, UK.10.1145/3583780.3614908 ↗
22. Arora, J., and Park, Y. (2023, January 9–14). Split-NER: Named entity recognition via two question-answering-based classifications. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Toronto, ON, Canada.10.18653/v1/2023.acl-short.36 ↗
23. Wang, H., Cheng, L., Zhang, W., Soh, D.W., and Bing, L. (2024, January 11–16). Order-Agnostic Data Augmentation for Few-Shot Named Entity Recognition. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand.10.18653/v1/2024.acl-long.421 ↗
24. Named Entity Recognition by Using XLNet-BiLSTM-CRF10.1007/s11063-021-10547-1 ↗
25. Cui, L., Wu, Y., Liu, J., Yang, S., and Zhang, Y. (2021). Template-Based Named Entity Recognition Using BART. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Association for Computational Linguistics.10.18653/v1/2021.findings-acl.161 ↗
26. Liao, X., Yuan, K., and Wang, X.F. (2016, January 24–28). Acing the IOC Game: Toward Automatic Discovery and Analysis of Open-Source Cyber Threat Intelligence. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria.10.1145/2976749.2978315 ↗
27. Zhu, Z., and Dumitras, T. (2018, January 24–26). Chainsmith: Automatically Learning the Semantics of Malicious Campaigns by Mining Threat Intelligence Reports. Proceedings of the 2018 IEEE European Symposium on Security and Privacy (EuroS&P), London, UK.10.1109/EuroSP.2018.00039 ↗
28. Zhou, S., Long, Z., and Tan, L. (2018, January 1–3). Automatic Identification of Indicators of Compromise Using Neural-Based Sequence Labelling. Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation, Hong Kong, China.
29. Dionísio, N., Alves, F., and Ferreira, P.M. (2019, January 4–19). Cyberthreat Detection from Twitter Using Deep Neural Networks. Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary.10.1109/IJCNN.2019.8852475 ↗
30. Named Entity Recognition in Threat Intelligence Domain Based on Deep Learning
31. Chen, S.-S., Hwang, R.-H., Sun, C.-Y., Lin, Y.-D., and Pai, T.-W. (2023, January 4–8). Enhancing Cyber Threat Intelligence with Named Entity Recognition Using BERT-CRF. Proceedings of the GLOBECOM 2023—2023 IEEE Global Communications Conference, Kuala Lumpur, Malaysia.10.1109/GLOBECOM54140.2023.10436853 ↗
32. Chen, J., Liu, Q., and Lin, H. (2022, January 22–27). Few-Shot Named Entity Recognition with Self-Describing Networks. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Dublin, Ireland.10.18653/v1/2022.acl-long.392 ↗
33. PADA: Example-Based Prompt Learning for On-the-Fly Adaptation to Unseen Domains10.1162/tacl_a_00468 ↗
34. Chen, X., Zhang, N., and Li, L. (2022, January 12–17). LightNER: A Lightweight Generative Framework with Prompt-Guided Attention for Low-Resource NER. Proceedings of the 29th International Conference on Computational Linguistics, Gyeongju, Republic of Korea.
35. Chinese Named Entity Recognition Based on Prompt Learning and Multi-Level Feature Fusion
36. Ye, F., Huang, L., Liang, S., and Chi, K. (2023). Decomposed Two-Stage Prompt Learning for Few-Shot Named Entity Recognition. Information, 14.10.3390/info14050262 ↗
37. MPE3: Learning Meta-Prompt with Entity-Enhanced Semantics for Few-Shot Named Entity Recognition10.1016/j.neucom.2024.129031 ↗
38. Tjong Kim Sang, E.F., and De Meulder, F. (June, January 31). Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, Edmonton, AB, Canada.10.3115/1119176.1119195 ↗
39. Linguistic Data Consortium (2006). ACE 2005 Multilingual Training Corpus, Linguistic Data Consortium.
40. Derczynski, L., Nichols, E., van Erp, M., and Limsopatham, N. (2017, January 7). Results of the WNUT2017 Shared Task on Novel and Emerging Entity Recognition. Proceedings of the 3rd Workshop on Noisy User-generated Text (W-NUT 2017), Copenhagen, Denmark.10.18653/v1/W17-4418 ↗
41. Weischedel, R., Pradhan, S., Ramshaw, L., Palmer, M., Xue, N., Marcus, M., Taylor, A., Greenberg, R., Hovy, E., and Maeda, K. (2013). OntoNotes Release 5.0, Linguistic Data Consortium. LDC2013T19.
42. VirusTotal (2025, September 03). VirusTotal: Online Malware Analysis. Available online: https://www.virustotal.com/.
43. MISP (2025, September 03). MISP: Open Source Threat Intelligence Platform. Available online: https://www.misp-project.org/.
44. OpenIOC (2025, September 03). OpenIOC: A Framework for Threat Intelligence Sharing. Available online: https://openioc.org/.
45. AlienVault OTX (2025, September 03). AlienVault Open Threat Exchange (OTX). Available online: https://otx.alienvault.com/.
46. Wang, X., Liu, X., and Ao, S. (January, January 29). DNR-TI: A Large-Scale Dataset for Named Entity Recognition in Threat Intelligence. Proceedings of the 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Guangzhou, China.10.1109/TrustCom50675.2020.00252 ↗
47. Automatic Extraction of Named Entities of Cyber Threats Using a Deep Bi-LSTM-CRF Network10.1007/s13042-020-01122-6 ↗
48. Wang, X., He, S., and Xiong, Z. (2022, January 4–6). APTNER: A Specific Dataset for NER Missions in Cyber Threat Intelligence Field. Proceedings of the 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Hangzhou, China.10.1109/CSCWD54268.2022.9776031 ↗
49. Bianco, D.J. (2024, March 29). The Pyramid of Pain. Enterprise Detection & Response Blog. Available online: http://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html.