Respite for SMEs: A Systematic Review of Socio-Technical Cybersecurity Metrics
Source abstract
Abstract
Cybersecurity threats are on the rise, and small- and medium-sized enterprises (SMEs) struggle to cope with these developments. To combat threats, SMEs must first be willing and able to assess their cybersecurity posture. Cybersecurity risk assessment, generally performed with the help of metrics, provides the basis for an adequate defense. Significant challenges remain, however, especially in the complex socio-technical setting of SMEs. Seemingly basic questions, such as how to aggregate metrics and ensure solution adaptability, are still open to debate. Aggregation and adaptability are vital topics to SMEs, as they require the assimilation of metrics into an actionable advice adapted to their situation and needs. To address these issues, we systematically review socio-technical cybersecurity metric research in this paper. We analyse aggregation and adaptability considerations and investigate how current findings apply to the SME situation. To ensure that we provide valuable insights to researchers and practitioners, we integrate our results in a novel socio-technical cybersecurity framework geared towards the needs of SMEs. Our framework allowed us to determine a glaring need for intuitive, threat-based cybersecurity risk assessment approaches for the least digitally mature SMEs. In the future, we hope our framework will help to offer SMEs some deserved respite by guiding the design of suitable cybersecurity assessment solutions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Information and Cyber Security · Network Security and Intrusion Detection · Supply Chain Resilience and Risk Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Information and Computing Sciences, Utrecht University, Princetonplein 5, 3584 CC Utrecht, The Netherlands
Department of Public Health and Primary Care, Leiden University Medical Center (LUMC), Albinusdreef 2, 2333 ZA Leiden, The Netherlands
Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Niels Bohrweg 1, 2333 CA Leiden, The Netherlands
Funding
Horizon 2020 Framework Programme
References (100+)
Show source-provided references
1. Bassett, G., Hylender, C.D., Langlois, P., Pinto, A., and Widup, S. (2021). Data Breach Investigations Report, Verizon. Technical Report.10.1016/S1361-3723(21)00061-0 ↗
2. Bissell, K., and Lasalle, R.M. (2019). Cost of Cybercrime Study, Accenture and Ponemon Institute. Technical Report.
3. Investigating the Security Divide between SME and Large Companies: How SME Characteristics Influence Organizational IT Security Investments10.1007/s10796-019-09959-1 ↗
4. Ponemon Institute (2019). Global State of Cybersecurity in Small and Medium-Sized Businesses, Ponemon Institute. Technical Report, Keeper Security.
5. Socio-Technical Systems Cybersecurity Framework10.1108/ICS-03-2018-0031 ↗
6. Carías, J.F., Arrizabalaga, S., Labaka, L., and Hernantes, J. (2020). Cyber Resilience Progression Model. Appl. Sci., 10.10.3390/app10217393 ↗
7. Advancing Socio-Technical Systems Thinking: A Call for Bravery10.1016/j.apergo.2013.02.009 ↗
8. Correlating Human Traits and Cyber Security Behavior Intentions10.1016/j.cose.2017.11.015 ↗
9. Drevin, L., Von Solms, S., and Theocharidou, M. (2020, January 21–23). Automating the Communication of Cybersecurity Knowledge: Multi-Case Study. Proceedings of the IFIP Advances in Information and Communication Technology, Maribor, Slovenia.
10. Calculated Risk? A Cybersecurity Evaluation Tool for SMEs10.1016/j.bushor.2020.03.010 ↗
11. Jaquith, A. (2007). Security Metrics: Replacing Fear, Uncertainty, and Doubt, Pearson Education.
12. Measuring Risk: Computer Security Metrics, Automation, and Learning10.1109/MAHC.2015.30 ↗
13. Modelling Adaptive Information Security for SMEs in a Cluster10.1108/JIC-05-2019-0128 ↗
14. European DIGITAL SME Alliance (2020). The EU Cybersecurity Act and the Role of Standards for SMEs—Position Paper, European DIGITAL SME Alliance. Technical Report.
15. Maturity Assessment and Process Improvement for Information Security Management in Small and Medium Enterprises10.1002/smr.1609 ↗
16. Organizational Characteristics Influencing SME Information Security Maturity
17. A Survey on Systems Security Metrics
18. STRAM: Measuring the Trustworthiness of Computer-Based Systems10.1145/3277666 ↗
19. Refsdal, A., Solhaug, B., and Stolen, K. (2015). Cyber-Risk Management, SpringerBriefs in Computer Science; Springer International Publishing.10.1007/978-3-319-23570-7 ↗
20. Eusgeld, I., Freiling, F.C., and Reussner, R. (2008). On Metrics and Measurements. Dependability Metrics: Advanced Lectures, Springer. Lecture Notes in Computer Science.10.1007/978-3-540-68947-8 ↗
21. Investigating and Comparing the Predictors of the Intention towards Taking Security Measures against Malware, Scams and Cybercrime in General10.1016/j.chb.2018.11.002 ↗
22. Gollmann, D., Herley, C., Koenig, V., Pieters, W., and Sasse, M.A. (2015). Socio-Technical Security Metrics. Dagstuhl Rep., 2015.
23. Selbst, A.D., Boyd, D., Friedler, S.A., Venkatasubramanian, S., and Vertesi, J. (2019, January 29–31). Fairness and Abstraction in Sociotechnical Systems. Proceedings of the Conference on Fairness, Accountability, and Transparency, Atlanta, GA, USA.10.1145/3287560.3287598 ↗
24. Taxonomy of Compliant Information Security Behavior10.1016/j.cose.2012.04.004 ↗
25. User Motivations in Protecting Information Security: Protection Motivation Theory Versus Self-Determination Theory10.1080/07421222.2017.1394083 ↗
26. Encouraging Information Security Behaviors in Organizations: Role of Penalties, Pressures and Perceived Effectiveness10.1016/j.dss.2009.02.005 ↗
27. Hiatt, J. (2006). ADKAR: A Model for Change in Business, Government, and Our Community, Prosci.
28. An Approach to Information Security Culture Change Combining ADKAR and the ISCA Questionnaire to Aid Transition to the Desired Culture10.1108/ICS-08-2017-0056 ↗
29. Verendel, V. (2009, January 8–11). Quantified Security Is a Weak Hypothesis: A Critical Survey of Results and Assumptions. Proceedings of the 2009 Workshop on New Security Paradigms Workshop, Oxford, UK.10.1145/1719030.1719036 ↗
30. Rudolph, M., and Schwarz, R. (2012, January 20–24). A Critical Survey of Security Indicator Approaches. Proceedings of the 2012 Seventh International Conference on Availability, Reliability and Security, Prague, Czech Republic.10.1109/ARES.2012.10 ↗
31. Survey of Attack Projection, Prediction, and Forecasting in Cyber Security10.1109/COMST.2018.2871866 ↗
32. Quantifiable & Comparable Evaluations of Cyber Defensive Capabilities: A Survey & Novel, Unified Approach10.1016/j.cose.2020.101907 ↗
33. DAG-Based Attack and Defense Modeling: Don’t Miss the Forest for the Attack Trees10.1016/j.cosrev.2014.07.001 ↗
34. Rocha, Á., and Pereira, R.P. (2020). Metrics and Indicators of Information Security Incident Management: A Systematic Mapping Study. Developments and Advances in Defense and Security, Springer.
35. A Survey of Cyber Security Management in Industrial Control Systems10.1016/j.ijcip.2015.02.002 ↗
36. Cybersecurity in Industrial Control Systems: Issues, Technologies, and Challenges10.1016/j.comnet.2019.106946 ↗
37. Eckhart, M., Brenner, B., Ekelhart, A., and Weippl, E. (2019). Quantitative Security Risk Assessment for Industrial Control Systems: Research Opportunities and Challenges. J. Internet Serv. Inf. Secur.
38. Security Data Collection and Data Analytics in the Internet: A Survey10.1109/COMST.2018.2863942 ↗
39. Sengupta, S., Chowdhary, A., Sabur, A., Alshamrani, A., Huang, D., and Kambhampati, S. (2020). A Survey of Moving Target Defenses for Network Security. IEEE Commun. Surv. Tutor.10.1109/COMST.2020.2982955 ↗
40. Game Theory for Network Security10.1109/SURV.2012.062612.00056 ↗
41. Model-Based Quantitative Network Security Metrics: A Survey10.1109/COMST.2017.2745505 ↗
42. A Review of Cyber Security Risk Assessment Methods for SCADA Systems10.1016/j.cose.2015.09.009 ↗
43. Mapping the Field of Software Life Cycle Security Metrics10.1016/j.infsof.2018.05.011 ↗
44. Unknown Vulnerability Risk Assessment Based on Directed Graph Models: A Survey10.1109/ACCESS.2019.2954092 ↗
45. Data Collection for Security Measurement in Wireless Sensor Networks: A Survey10.1109/JIOT.2018.2883403 ↗
46. Ray, J., Marshall, H., De Sousa, V., Jean, J., Warren, S., and Bachand, S. (2020, November 16). Cyber Threatscape Report. Available online: https://www.accenture.com/us-en/insights/security/cyber-threatscape-report.
47. Threat-Based Risk Assessment for Enterprise Networks
48. Ferguson, N., and Schneier, B. (2003). Practical Cryptography, Wiley.
49. Lippmann, R.P., Riordan, J.F., Yu, T., and Watson, K.K. (2012). Continuous Security Metrics for Prevalent Network Threats: Introduction and First Four Metrics, Massachusetts Institute of Technology Lexington Lincoln Lab. Technical Report.10.21236/ADA565825 ↗
50. A Hybrid Information Security Risk Assessment Procedure Considering Interdependences between Controls10.1016/j.eswa.2011.07.015 ↗
51. Information Security Management: ANP Based Approach for Risk Analysis and Decision Making
52. Physical Intrusion Games—Optimizing Surveillance by Simulation and Game Theory10.1109/ACCESS.2017.2693425 ↗
53. Evesti, A., and Ovaska, E. (2013). Comparison of Adaptive Information Security Approaches. ISRN Artif. Intell., 2013.10.1155/2013/482949 ↗
54. An Analytics Approach to Adaptive Maturity Models Using Organizational Characteristics10.1186/s40165-016-0022-1 ↗
55. Corporate Security Solutions for BYOD: A Novel User-Centric and Self-Adaptive System10.1016/j.comcom.2015.07.019 ↗
56. A Taxonomy of Dirty Data10.1023/A:1021564703268 ↗
57. Learning in the Presence of Concept Drift and Hidden Contexts10.1007/BF00116900 ↗
58. van Haastrecht, M., Sarhan, I., Yigit Ozkan, B., Brinkhuis, M., and Spruit, M. (2021). SYMBALS: A Systematic Review Methodology Blending Active Learning and Snowballing. Front. Res. Metrics Anal., 6.10.3389/frma.2021.685591 ↗
59. Kitchenham, B., and Charters, S. (2007). Guidelines for Performing Systematic Literature Reviews in Software Engineering, Keele University and Durham University Joint Report. Technical Report.
60. The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration10.1016/j.jclinepi.2009.06.006 ↗
61. Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 Statement10.1186/2046-4053-4-1 ↗
62. An Open Source Machine Learning Framework for Efficient and Transparent Systematic Reviews10.1038/s42256-020-00287-7 ↗
63. Wohlin, C. (2014, January 13–14). Guidelines for Snowballing in Systematic Literature Studies and a Replication in Software Engineering. Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, London, UK.10.1145/2601248.2601268 ↗
64. On the Performance of Hybrid Search Strategies for Systematic Literature Reviews in Software Engineering10.1016/j.infsof.2020.106294 ↗
65. Pybliometrics: Scriptable Bibliometrics Using a Python Interface to Scopus10.1016/j.softx.2019.100263 ↗
66. Harrison, H., Griffin, S.J., Kuhn, I., and Usher-Smith, J.A. (2020). Software Tools to Support Title and Abstract Screening for Systematic Reviews in Healthcare: An Evaluation. BMC Med Res. Methodol., 20.10.1186/s12874-020-0897-3 ↗
67. Lessons from Applying the Systematic Literature Review Process within the Software Engineering Domain10.1016/j.jss.2006.07.009 ↗
68. Measuring Security10.1109/MSP.2011.56 ↗
69. Noel, S., and Jajodia, S. (2014, January 8–10). Metrics Suite for Network Attack Graph Analytics. Proceedings of the 9th Annual Cyber and Information Security Research Conference, Oak Ridge, TN, USA.10.1145/2602087.2602117 ↗
70. Spruit, M., and Roeling, M. (2014, January 9–11). ISFAM: The Information Security Focus Area Maturity Model. Proceedings of the ECIS 2014, Atlanta, GA, USA.
71. Security Events and Vulnerability Data for Cybersecurity Risk Estimation10.1111/risa.12864 ↗
72. Cormack, G.V., and Grossman, M.R. (2016, January 17–21). Engineering Quality and Reliability in Technology-Assisted Review. Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, Pisa, Italy.10.1145/2911451.2911510 ↗
73. Zhou, Y., Zhang, H., Huang, X., Yang, S., Babar, M.A., and Tang, H. (2015, January 27–29). Quality Assessment of Systematic Reviews in Software Engineering: A Tertiary Study. Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering, Nanjing, China.10.1145/2745802.2745815 ↗
74. Information Security Assessment and Reporting: Distributed Defense
75. Rhee, K.H., and Yi, J.H. (2015). Advanced Security Assessment for Control Effectiveness. Information Security Applications, Springer International Publishing. Lecture Notes in Computer Science.10.1007/978-3-319-15087-1 ↗
76. Yang, N., Singh, T., and Johnston, A. (2020). A Replication Study of User Motivation in Protecting Information Security Using Protection Motivation Theory and Self Determination Theory. AIS Trans. Replication Res., 6.
77. Tryfonas, T., and Askoxylakis, I. (2014). DSAPE—Dynamic Security Awareness Program Evaluation. Human Aspects of Information Security, Privacy, and Trust, Springer International Publishing. Lecture Notes in Computer Science.
78. Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vulnerabilities10.1109/TSE.2010.81 ↗
79. A Vulnerability Life Cycle-Based Security Modeling and Evaluation Approach10.1093/comjnl/bxs112 ↗
80. Fuzzy Multi-Criteria Decision-Making for Information Security Risk Assessment10.2174/1874110X01206010026 ↗
81. A Multidimensional Approach to Information Security Risk Management Using FMEA and Fuzzy Theory10.1016/j.ijinfomgt.2014.07.005 ↗
82. Assessing Information Security Risk for an Evolving Smart City Based on Fuzzy and Grey FMEA10.3233/JIFS-172097 ↗
83. Kantor, P., Muresan, G., Roberts, F., Zeng, D.D., Wang, F.Y., Chen, H., and Merkle, R.C. (2005). Risk Management Using Behavior Based Bayesian Networks. Intelligence and Security Informatics, Springer. Lecture Notes in Computer Science.10.1007/b136511 ↗
84. An Input–Output Measurable Design for the Security Meter Model to Quantify and Manage Software Security Risk10.1109/TIM.2007.915139 ↗
85. Network Risk Management Using Attacker Profiling10.1002/sec.58 ↗
86. A Security Risk Analysis Model for Information Systems: Causal Relationships of Risk Factors and Vulnerability Propagation Analysis10.1016/j.ins.2013.02.036 ↗
87. A Hybrid Delphi-Bayesian Method to Establish Business Data Integrity Policy: A Benchmark Data Center Case Study10.1108/03684921011043260 ↗
88. Information Security Risk Modeling Using Bayesian Index10.1093/comjnl/bxq059 ↗
89. Abramowicz, W., and Paschke, A. (2018). Information Security Management Systems—A Maturity Model Based on ISO/IEC 27001. Business Information Systems, Springer International Publishing. Lecture Notes in Business Information Processing.10.1007/978-3-319-93931-5 ↗
90. PenQuest: A Gamified Attacker/Defender Meta Model for Cyber Security Assessment and Education10.1007/s11416-019-00342-x ↗
91. Validation of a Socio-Technical Management Process for Optimising Cybersecurity Practices10.1016/j.cose.2020.101846 ↗
92. Systematic Approach to Cyber Resilience Operationalization in SMEs10.1109/ACCESS.2020.3026063 ↗
93. Design and Validation of Information Security Culture Framework10.1016/j.chb.2015.03.054 ↗
94. Defining Organisational Information Security Culture—Perspectives from Academia and Industry10.1016/j.cose.2020.101713 ↗
95. A Socio-Technical Approach to Preventing, Mitigating, and Recovering from Ransomware Attacks10.4338/ACI-2016-04-SOA-0064 ↗
96. Clarke, N., and Furnell, S. (2020, January 8–10). Addressing SME Characteristics for Designing Information Security Maturity Models. Proceedings of the IFIP Advances in Information and Communication Technology, Mytilene, Greece.
97. Cultivating Cybersecurity Learning: An Integration of Self-Determination and Flow10.1016/j.cose.2020.101875 ↗
98. Moving from a “human-as-Problem” to a “human-as-Solution” Cybersecurity Mindset10.1016/j.ijhcs.2019.05.005 ↗
99. Liu, Y., Sarabi, A., Zhang, J., Naghizadeh, P., Karir, M., Bailey, M., and Liu, M. (2015, January 12–14). Cloudy with a Chance of Breach: Forecasting Cyber Security Incidents. Proceedings of the 24th {USENIX} Security Symposium ({USENIX} Security 15), Washington, DC, USA.
100. Toward the Automation of Threat Modeling and Risk Assessment in IoT Systems10.1016/j.iot.2019.100056 ↗