A Flexible Risk-Based Security Evaluation Methodology for Information Communication Technology System Certification
Source abstract
Abstract
As Information and Communication Technology (ICT) systems become increasingly complex, the need for adaptable and efficient security certification frameworks grows. This paper introduces a flexible security evaluation methodology designed to serve as the foundation for cybersecurity certification across diverse ICT systems. The proposed methodology integrates risk assessment and test-based evaluation, offering a scalable approach that adapts to different tools and processes, addressing the limitations of existing rigid certification schemes. The certification approach expands on ETSI’s Risk-Based Security Assessment and Testing methods, based on ISO 31000 and ISO 29119, and it integrates widely recognized standards such as MUD. This ensures an objective, empirical evaluation process that enables partial automation and simplifies recertification. As a proof of concept, we validate the methodology in two real use cases, an ICT gateway for smart grids and an AI-powered investments platform, demonstrating its flexibility and applicability to real-world contexts while addressing the challenges of modern ICT ecosystems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Information and Cyber Security · Advanced Malware Detection Techniques · Network Security and Intrusion Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Information and Communication Engineering, Computer Science Faculty, University of Murcia, 30100 Murcia, Spain
ResilTech s.r.l., Piazza Nilde Iotti, 25, 56025 Pontedera, Italy
7bulls.com, Aleja Armii Ludowej 26, 00-609 Warszawa, Poland
Funding
European Commission · DOSS
References (46)
Show source-provided references
1. Gartner (2025, February 03). Top Cybersecurity Trends and Strategies for Securing the Future. Available online: https://www.gartner.com/en/cybersecurity/topics/cybersecurity-trends.
2. Regulation (EU) 2019/881 of the European Parliament and of the Council of 17 April 2019 on ENISA (the European Union Agency for Cybersecurity) and on information and communications technology cybersecurity certification and repealing Regulation (EU) No 526/2013 (Cybersecurity Act) (Text with EEA relevance), 2013.
3. Regulation (EU) 2024/2847 of the European Parliament and of the Council of 23 October 2024 on horizontal cybersecurity requirements for products with digital elements and amending Regulations (EU) No 168/2013 and (EU) 2019/1020 and Directive (EU) 2020/1828 (Cyber Resilience Act) (Text with EEA relevance), 2024.
4. European Telecommunications Standards Institute (ETSI) (2025, February 03). ETSI EG 203 251 V1.1.1: Methods for Testing and Specification; Risk-based Security Assessment and Testing Methodologies. Available online: https://www.etsi.org/deliver/etsi_eg/203200_203299/203251/01.01.01_60/eg_203251v010101p.pdf.
5. (2018). Risk Management—Guidelines (Standard No. ISO 31000:2018).
6. (2021). Software and Systems Engineering—Software Testing. Part 2: Test Processes (Standard No. ISO/IEC/IEEE 29119-2:2021).
7. Lear, E., Romascanu, D., and Droms, R. (2025, February 03). Manufacturer Usage Description Specification. Standard IETF RFC 8520. Available online: https://tools.ietf.org/html/rfc8520.
8. Alberts, C.J., Dorofee, A.J., Stevens, J.F., and Woody, C. (2025, February 03). OCTAVE-S Implementation Guide, Version 1; Carnegie Mellon University. Technical Report; 2005. Available online: https://insights.sei.cmu.edu/documents/1608/2005_002_001_14273.pdf.
9. E-Safety Vehicle Intrusion Protected Applications (EVITA) (2025, February 03). D3.4.3-On-Board Architecture and Protocols Verification; 2010. Available online: https://www.evita-project.org/deliverables.html.
10. Taxonomy of information security risk assessment (ISRA)10.1016/j.cose.2015.11.001 ↗
11. A Survey of Cybersecurity Certification for the Internet of Things
12. Toward a Cybersecurity Certification Framework for the Internet of Things10.1109/MSEC.2019.2904475 ↗
13. European Cyber Security Organisation (ECSO), WG1—Standardisation, Certification, Labelling and Supply Chain Management (2025, February 03). Overview of Existing Cybersecurity Standards and Certification Schemes v2; 2017. Available online: https://ecs-org.eu/ecso-uploads/2022/10/5a31129ea8e97.pdf.
14. Common Criteria (2025, February 03). Common Criteria for Information Technology Security Evaluation. Part 1: Introduction and General Model; 2022. Available online: https://commoncriteriaportal.org/files/ccfiles/CC2022PART1R1.pdf.
15. National Cyber Security Centre (NCSC) (2025, February 03). The Commercial Product Assurance (CPA) Build Standard v1.4; NCSC-1844117881-312, Available online: https://www.ncsc.gov.uk/files/CPA-Build_Standard_1-4.pdf.
16. Underwriters Laboratories (UL) (2025, February 03). Software Cybersecurity for Network-Connectable Products, Part 2-1: Particular Requirements for Network Connectable Components of Healthcare and Wellness Systems; 2023. Available online: https://www.shopulstandards.com/ProductDetail.aspx?productId=UL2900-2-1.
17. Agence Nationale de la séCurité des Systèmes D’information (ANSSI) (2025, February 03). Certification de séCurité de Premier Niveau des Produits des Technologies de L’information (CSPN); Paris, No. 45 /ANSSI/SDE/PSS/CCN; 2023. Available online: https://cyber.gouv.fr/sites/default/files/document/ANSSI-CSPN-CER-P-01%20Certification_de_securite_de_premier_niveau_v5.0.pdf.
18. Zhou, C., and Ramacciotti, S. (2011). Common Criteria: Its Limitations and Advice on Improvement. ISSA J., 24–28. Available online: https://www.difesa.it/assets/allegati/33182/commoncriteria_issa_journal_0411.pdf.
19. The Challenges of Software Cybersecurity Certification [Building Security In]10.1109/MSEC.2020.3037845 ↗
20. Fowler, D., Epiphaniou, G., and Maple, C. (2025, February 03). Cybersecurity Assurance and Certification for Systems. Available online: https://www.researchgate.net/publication/370873334_Cybersecurity_Assurance_and_Certification_for_Systems.
21. EU Cybersecurity Act and IoT Certification: Landscape, Perspective and a Proposed Template Scheme10.1109/ACCESS.2022.3225973 ↗
22. Defining the Behavior of IoT Devices Through the MUD Standard: Review, Challenges, and Research Directions10.1109/ACCESS.2021.3111477 ↗
23. Integrating the manufacturer usage description standard in the modelling of cyber–physical systems10.1016/j.csi.2023.103777 ↗
24. (2018). Road Vehicles—Functional Safety Part 3: Concept Phase (Standard No. ISO 26262-3:2018).
25. Failure Modes and Effects Analysis (FMEA) (2025, February 03). University of Cambdridge. Available online: https://www.ifm.eng.cam.ac.uk/research/dmg/tools-and-techniques/fmea-failure-modes-and-effects-analysis/.
26. (2009). Business Continuity Management (Standard No. BSI-Standard 100-4).
27. Federal Office for Information Security (BSI) (2011). Privacy Impact Assessment Guideline, Federal Office for Information Security (BSI).
28. National Institute of Standards and Technology (NIST) (2025, February 03). National Vulnerability Database (NVD), Available online: https://nvd.nist.gov.
29. Matheu, S., Sánchez, A., Cioroaica, E., Daoudagh, S., Lonetti, F., Marchetti, E., Schiavone, E., Massimiliano, L., Sorokos, I., and Pintos, B. (2025, February 03). D7.1 Report on the Identified Security and Privacy Metrics and Security Claims to Evaluate the Security of a System. BIECO Project—Building Trust in Ecosystems and Ecosystem Components. Available online: https://www.bieco.org/project-description/deliverables/d7-1-report-on-the-identified-security-and-privacy-metrics-and-security-claims-to-evaluate-the-security-of-a-system.
30. National Telecommunications and Information Administration (NTIA) (2025, February 03). Software Bill of Materials (SBOM), Available online: https://ntia.gov/SBOM.
31. (2023). Methods for Testing and Specification (MTS); Testing and Test Control Notation Version 3; Part 1: TTCN-3 Core Language (Standard No. ETSI ES 201 873-1).
32. (2012). Joint Task Force Transformation Initiative. Guide for Conducting Risk Assessments: Information security (Standard No. NIST SP 800-30).
33. Lautenbach, A. (2025, February 03). D2. Security Models. HEAVENS project—HEAling Vulnerabilities to ENhance Software Security and Safety. Available online: https://autosec.se/wp-content/uploads/2018/03/HEAVENS_D2_v2.0.pdf.
34. Eichler, J., and Angermeier, D. (2015, January 21–22). Modular risk assessment for the development of secure automotive systems. Proceedings of the 31st VDI/VW Joint Conference Automotive Security, Wolfsburg, Germany.
35. European Cyber Security Organisation (ECSO) (2025, February 03). Cybersecurity Made in Europe. Available online: https://www.cybersecurity-label.eu/.
36. IoT Security and Privacy Label (2025, February 03). University of Carnegie Mellon. Available online: https://www.iotsecurityprivacy.org/.
37. Directive 2010/30/EU of the European Parliament and of the Council of 19 May 2010 on the indication by labelling and standard product information of the consumption of energy and other resources by energy-related products (recast) (Text with EEA relevance. Directive-2010/30-EN-EUR-Lex. 2010.
38. National Institute of Standards and Technology (NIST) (2025, February 03). Considerations for Managing Internet of Things (IoT) Cybersecurity and Privacy Risks, Available online: https://nvlpubs.nist.gov/nistpubs/ir/2018/NIST.IR.8228-draft.pdf.
39. Net2DG Project-Leveraging NETworked Data for the Digital Electricity Grid (2025, February 03). Funded by European Union’s Horizon 2020 Research and Innovation Programme Under Grant Agreement No 774145. Available online: http://www.net2dg.eu/.
40. Ben-Kiki, O., Evans, C., and Ingy, N. (2025, February 03). YAML Ain’t Markup Language (YAML) Version 1.2. Available online: https://yaml.org/spec/1.2.2/.
41. Schiavone, E., Nostro, N., and Brancati, F. (2021, January 22–26). A MDE Tool for Security Risk Assessment of Enterprises. Proceedings of the 10th Latin-American Symposium on Dependable Computing (LADC 2021). Sociedade Brasileira de Computação, Porto Alegre, Brasil.10.5753/ladc.2021.18530 ↗
42. Risk-based automated assessment and testing for the cybersecurity certification and labelling of IoT devices10.1016/j.csi.2018.08.003 ↗
43. Lorrain, J., Fourneret, E., Dadeau, F., and Legeard, B. (2025, February 03). MBeeTle-un Outil Pour la géNération de Tests à-la-volée à l’aide de Modèles. Groupement De Recherche CNRS du Génie de la Programmation et du Logiciel, Besançon, France. 2016, Available online: https://hal.science/hal-02472608.
44. Bicchierai, I., Araniti, E., Matheu-García, S.N., and Gil, J.F.M. (2023, January 9–12). Validating the BIECO Security Evaluation Methodology within a Smart Grid Monitoring SW. Proceedings of the 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW), Florence, Italy.10.1109/ISSREW60843.2023.00069 ↗
45. DAEMON: A Domain-Based Monitoring Ontology for IoT Systems10.1007/s42979-023-01975-y ↗
46. Matheu, S., Sebastio, S., Skarmeta, A., Orizio, R., Vasileiadis, S., Kalos, V., Muller, K., Grubl, T., Omana, R., and Tuck, S. (2024, January 26–28). Active Security for Connected Device Lifecycle: The CERTIFY Architecture. Proceedings of the Special Session on Intelligent Internet of Things Security and Privacy (WISP), Salamanca, Spain.