Quality Dimensions for Automatic Assessment of Structured Cyber Threat Intelligence Data
Source abstract
Abstract
Cyber threat intelligence (CTI) has emerged as a promising approach to mitigating the effect of malicious activities. However, the potential usability of CTI data depends largely on their quality. The available CTI data quality assessment methods are either not fully automatic or deliver just a few dimensions. In this paper, we propose an automated CTI data quality assessment method that separately provides an assessment of CTI contents and confidence scores of CTI providers. Specifically, we introduce new dimensions to accommodate the requirements of the technical and tactical levels of CTI data. A comprehensive CTI quality assessment is proposed on CTI data provided in structured STIX 2.1 notation. Moreover, we present a visualization of the results to more easily interpret the obtained values of the quality dimensions. Extensive experiments on real datasets demonstrate that our proposed method can quantitatively and efficiently assess CTI data quality.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Intelligence, Security, War Strategy · Bayesian Modeling and Causal Inference · Information and Cyber Security
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, Kaunas University of Technology, LT-51390 Kaunas, Lithuania
Funding
Economic Revitalization and Resilience Enhancement Plan “New Generation Lithuania”
References (51)
Show source-provided references
1. Chatziamanetoglou, D., and Rantos, K. (2024). Cyber Threat Intelligence on Blockchain: A Systematic Literature Review. Computers, 13.10.3390/computers13030060 ↗
2. Current approaches and future directions for Cyber Threat Intelligence sharing: A survey
3. Sharing Machine Learning Models as Indicators of Compromise for Cyber Threat Intelligence10.3390/jcp1010008 ↗
4. Bandara, E., Shetty, S., Mukkamala, R., Rahaman, A., and Liang, X. (2022, January 18–20). LUUNU—Blockchain, MISP, Model Cards and Federated Learning Enabled Cyber Threat Intelligence Sharing Platform. Proceedings of the 2022 Annual Modeling and Simulation Conference (ANNSIM), San Diego, CA, USA.10.23919/ANNSIM55834.2022.9859355 ↗
5. Efficient collective action for tackling time-critical cybersecurity threats10.1093/cybsec/tyad021 ↗
6. DEALER: Decentralized incentives for threat intelligence reporting and exchange10.1007/s10207-020-00528-1 ↗
7. Ali, H., Ahmad, J., Jaroucheh, Z., Papadopoulos, P., Pitropakis, N., Lo, O., Abramson, W., and Buchanan, W.J. (2022). Trusted Threat Intelligence Sharing in Practice and Performance Benchmarking through the Hyperledger Fabric Platform. Entropy, 24.10.3390/e24101379 ↗
8. Cyber Threat Intelligence Mining for Proactive Cybersecurity Defense: A Survey and New Perspectives10.1109/COMST.2023.3273282 ↗
9. Blockchain-Based Cyber Threat Intelligence Sharing Using Proof-of-Quality Consensus10.1155/2023/3303122 ↗
10. Cyber Threat Intelligence Sharing for Co-Operative Defense in Multi-Domain Entities10.1109/TDSC.2022.3214423 ↗
11. Dimitriadis, A., Lontzetidis, E., and Mavridis, I. (2021, January 26–28). Evaluation and Enhancement of the Actionability of Publicly Available Cyber Threat Information in Digital Forensics. Proceedings of the 2021 IEEE International Conference on Cyber Security and Resilience (CSR), Rhodes, Greece.10.1109/CSR51186.2021.9527934 ↗
12. Barnum, S. (2024, October 18). Standardizing Cyber Threat Intelligence Information with the Structured Threat Information Expression (STIX™). Available online: https://stixproject.github.io/about/STIX_Whitepaper_v1.1.pdf.
13. Cyber-threat intelligence for security decision-making: A review and research agenda for practice10.1016/j.cose.2023.103352 ↗
14. (2024, October 18). Introduction to STIX. Available online: https://oasis-open.github.io/cti-documentation/stix/intro.html.
15. Beyond accuracy: What data quality means to data consumers10.1080/07421222.1996.11518099 ↗
16. (2008). Software Engineering-Software Product Quality Requirements and Evaluation (Square)-Data Quality Model (Standard No. ISO/IEC 25012). Available online: https://www.iso.org/standard/35736.html.
17. A product perspective on total data quality management10.1145/269012.269022 ↗
18. (2015). Systems and Software Engineering—Systems and Software Quality Requirements and Evaluation (SQuaRE)—Measurement of Data Quality (Standard No. ISO 25024). Available online: https://www.iso.org/standard/35749.html.
19. Gao, Y., Li, X., Li, J., Gao, Y., and Guo, N. (2018, January 10–13). Graph Mining-based Trust Evaluation Mechanism with Multidimensional Features for Large-scale Heterogeneous Threat Intelligence. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.10.1109/BigData.2018.8622111 ↗
20. Li, Q., Jiang, Z., Yang, Z., Liu, B., Wang, X., and Zhang, Y. (2018, January 1–3). A Quality Evaluation Method of Cyber Threat Intelligence in User Perspective. Proceedings of the 2018 17th IEEE International Conference on Trust, Security and Privacy in Computing and Communications/12th IEEE International Conference on Big Data Science and Engineering (TrustCom/BigDataSE), New York, NY, USA.10.1109/TrustCom/BigDataSE.2018.00049 ↗
21. Meier, R., Scherrer, C., Gugelmann, D., Lenders, V., and Vanbever, L. (June, January 29). FeedRank: A tamper-resistant method for the ranking of cyber threat intelligence feeds. Proceedings of the 2018 10th International Conference on Cyber Conflict (CyCon), Tallinn, Estonia.10.23919/CYCON.2018.8405024 ↗
22. Li, V.G., Dunn, M., Pearce, P., McCoy, D., Voelker, G.M., and Savage, S. (2019, January 14–16). Reading the tea leaves: A comparative analysis of threat intelligence. Proceedings of the 28th USENIX Security Symposium (USENIX Security 19), Santa Clara, CA, USA.
23. Schaberreiter, T., Kupfersberger, V., Rantos, K., Spyros, A., Papanikolaou, A., Ilioudis, C., and Quirchmayr, G. (2019, January 26–29). A quantitative evaluation of trust in the quality of cyber threat intelligence sources. Proceedings of the 14th international Conference on Availability, Reliability and Security.10.1145/3339252.3342112 ↗
24. Measuring and visualizing cyber threat intelligence quality10.1007/s10207-020-00490-y ↗
25. Mavzer, K.B., Konieczna, E., Alves, H., Yucel, C., Chalkias, I., Mallis, D., Cetinkaya, D., and Sanchez LA, G. (2021, January 26–28). Trust and QualityComputation for Cyber Threat Intelligence Sharing Platforms. Proceedings of the 2021 IEEE International Conference on Cyber Security and Resilience (CSR), Rhodes, Greece.10.1109/CSR51186.2021.9527975 ↗
26. Threat intelligence quality dimensions for research and practice. Digit
27. An automated dynamic quality assessment method for cyber threat intelligence10.1016/j.cose.2024.104079 ↗
28. Improving quality of indicators of compromise using STIX graphs10.1016/j.cose.2024.103972 ↗
29. A Methodology for Developing & Assessing CTI Quality Metrics10.1109/ACCESS.2024.3351108 ↗
30. Grispos, G., Glisson, W.B., and Storer, T. (2019, January 8–11). How good is your data? Investigating the quality of data generated during security incident response investigations. Proceedings of the 52nd Hawaii International Conference on System Sciences Scholar Space Hawaii International, Maui, HI, USA. Available online: https://hdl.handle.net/10125/60152.10.24251/HICSS.2019.859 ↗
31. Dalziel, H. (2015). A Problem Well-Defined is Half-Solved. How to Define and Build an Effective Cyber Threat Intelligence Capability, Elsevier.10.1016/B978-0-12-802730-1.00002-8 ↗
32. Krzhizhanovskaya, V.V., Závodszky, G., Lees, M.H., Dongarra, J.J., Sloot, P.M.A., Brissos, S., and Teixeira, J. (2020). On the Automated Assessment of Open-Source Cyber Threat Intelligence Sources. Lecture Notes in Computer Science, Proceedings of the International Conference on Computational Science, Amsterdam, The Netherlands, 3–5 June 2020, Springer.
33. Bringas, P.G., García, H.P., de Pisón, F.J.M., Flecha, J.R.V., Lora, A.T., de la Cal, E.A., Herrero, Á., Álvarez, F.M., and Psaila, G. (2023). Measuring the Quality Information of Sources of Cybersecurity by Multi-Criteria Decision Making Techniques. Lecture Notes in Computer Science, Proceedings of the Hybrid Artificial Intelligent Systems, Salamancaa, Spain, 5–7 September 2022, Springer.
34. Wang, M., Yang, L., and Lou, W. (2022, January 27–30). A Comprehensive Dynamic Quality Assessment Method for Cyber Threat Intelligence. Proceedings of the 2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), Baltimore, MD, USA.10.1109/DSN-W54100.2022.00037 ↗
35. Griffioen, H., Booij, T., and Doerr, C. (2020, January 19–22). Quality evaluation of cyber threat intelligence feeds. Proceedings of the International Conference on Applied Cryptography and Network Security: 18th International Conference, ACNS 2020, Rome, Italy. Proceedings, Part II.10.1007/978-3-030-57878-7_14 ↗
36. Ahlemann, F., and Gastl, H. (2007). Process model for an empiracally grounded reference model construction. Reference Modeling for Business Systems Analysis, IGI Global Scientific Publishing.10.4018/978-1-59904-054-7.ch004 ↗
37. Sakellariou, G., Fouliras, P., Mavridis, I., and Sarigiannidis, P. (2022). A Reference Model for Cyber Threat Intelligence (CTI) Systems. Electronics, 11.10.3390/electronics11091401 ↗
38. A Feature-driven Method for Automating the Assessment of OSINT Cyber Threat Sources10.1016/j.cose.2021.102576 ↗
39. A Reliability Comparison Method for OSINT Validity Analysis10.1109/TII.2018.2857213 ↗
40. Methodologies for data quality assessment and improvement10.1145/1541880.1541883 ↗
41. (2024, October 28). European Union Agency for Cybersecurity, Actionable Information for Security Incident Response. Available online: https://www.enisa.europa.eu/publications/actionable-information-for-security.
42. Geras, T., and Schreck, T. (October, January 30). The “Big Beast to Tackle”: Practices in Quality Assurance for Cyber Threat Intelligence. Proceedings of the RAID ‘24: 27th International Symposium on Research in Attacks, Intrusions and Defenses, Padua, Italy.
43. Dziech, A., Mees, W., and Czyżewski, A. (2020). On the Assessment of Completeness and Timeliness of Actionable Cyber Threat Intelligence Artefacts. Communications in Computer and Information Science, Proceedings of the Communications in Computer and Information Science, Kraków, Poland, 8–9 October 2020, Springer.
44. Investigating Sharing of Cyber Threat Intelligence and Proposing a New Data Model for Enabling Automation in Knowledge Representation and Exchange
45. (2024, October 28). Cyber Threat Intelligence (CTI) Quality Metrics. Available online: https://github.com/geosakel77/s2.
46. (2024, October 28). AlienVault Open Threat Exchange (OTX). Available online: https://otx.alienvault.com/dashboard/new.
47. (2024, October 28). Mitre-Attack/Attack-Stix-Data. Available online: https://github.com/mitre-attack/attack-stix-data/.
48. (2024, September 27). Google/Gson. Available online: https://github.com/google/gson.
49. (2024, October 07). Apache POI. Available online: https://poi.apache.org/download.html.
50. (2024, October 28). Indicator for Malicious URL. Available online: https://oasis-open.github.io/cti-documentation/examples/indicator-for-malicious-url.html.
51. Venčkauskas, A., Jusas, V., Barisas, D., and Misnevs, B. (2024). Blockchain-Based Model for Incentivized Cyber Threat Intelligence Sharing. Appl. Sci., 14.10.3390/app14166872 ↗