Evaluation Methodology for Circular and Resilient Information Systems
Source abstract
Abstract
Digital technologies nowadays provide essential support for companies, making them a priority for businesses and a prominent area of study for researchers. In response to the increasing emphasis on sustainability and resilience, new information systems are developing to meet evolving business needs, namely circular and resilient information systems (CRISs). These systems integrate with traditional ones to optimise key performance indicators (KPIs) related to circularity and resiliency. Despite extensive methodologies for evaluating traditional information systems, systems designed for circularity and resiliency need to be assessed in parallel and in depth. Existing evaluations focus on efficiency and user satisfaction but often neglect the unique demands of circularity and resiliency. This study introduces a novel evaluation methodology for CRISs. Through a case study of an innovative system and the established literature, we address real-life needs and challenges in manufacturing. In particular, the system serves the needs of three distinct case studies: Carbon Fibre-Reinforced Polymer (CFRP) waste utilisation in drone manufacturing, recovery of magnets from Waste Electrical and Electronic Equipment (WEEE), and the repurposing of citrus processing waste into juice by-products. Our methodology is built on the 5W1H method to make our approach context-specific and aligned with each case’s unique requirements, making it also replicable for other industries. Our findings offer insights and a tool for practitioners and researchers to evaluate CRIS performance. The research highlights the importance of a two-fold evaluation approach for CRISs, evaluating both pilot-specific KPIs and the system’s technical performance. Policy implications suggest the need for regulatory frameworks and incentives to support the adoption, as well as evaluation, of CRISs and promote sustainable and resilient industrial practices.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software System Performance and Reliability · Advanced Software Engineering Methodologies · Green IT and Sustainability
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
ELTRUN—Τhe E-Business Research Center, Department of Management Science and Technology, Athens University of Economics and Business, Patission 76, 10434 Athens, Greece
Industrial and Digital Innovations Research Group—Indigo, School of Production Engineering and Management, Technical University of Crete, 73100 Chania, Greece
Funding
European Union’s Horizon 2020 research and innovation programme
References (45)
Show source-provided references
1. Thürer, M., Riedel, R., von Cieminski, G., and Romero, D. (2024). Developing a Circular and Resilient Information System: A Design Science Approach. Advances in Production Management Systems. Production Management Systems for Volatile, Uncertain, Complex, and Ambiguous Environments, Springer.10.1007/978-3-031-65894-5 ↗
2. Mobilising Information Systems Scholarship for a Circular Economy: Review, Synthesis, and Directions for Future Research10.1111/isj.12305 ↗
3. Aspects of Resilience for Smart Manufacturing Systems10.1002/jsc.2555 ↗
4. Qualitative Methods Research in Information Systems: Motivations, Themes, and Contributions10.1057/ejis.2011.57 ↗
5. Circularity Information Platform for the Built Environment10.1016/j.autcon.2023.104933 ↗
6. Reliability Non-Functional Requirement Evaluation in Mission-Critical Systems with an Architectural Strategy for Future Systems
7. Wynn, M., and Jones, P. (2022). Digital Technology Deployment and the Circular Economy. Sustainability, 14.10.3390/su14159077 ↗
8. A Conceptual Framework for Resilience: Fundamental Definitions, Strategies and Metrics10.1007/s00607-020-00874-x ↗
9. Integrating ERP and MFCA Systems for Improved Waste-Reduction Decisions in a Brewery in South Africa10.1016/j.jclepro.2012.09.013 ↗
10. A Large Multi-Group Decision-Making Technique for Prioritising the Big Data-Driven Circular Economy Practices in the Automobile Component Manufacturing Industry10.1016/j.techfore.2020.120567 ↗
11. Zero Waste Manufacturing: A Framework and Review of Technology, Research, and Implementation Barriers for Enabling a Circular Economy Transition in Singapore10.1016/j.resconrec.2019.104438 ↗
12. The Information Resilience Framework: Vulnerabilities, Capabilities, and Requirements10.1145/3388786 ↗
13. Empirical Investigation of Data Analytics Capability and Organizational Flexibility as Complements to Supply Chain Resilience10.1080/00207543.2019.1582820 ↗
14. Ihlenfeldt, S., Wunderlich, T., Süße, M., Hellmich, A., Schenke, C.-C., Wenzel, K., and Mater, S. (2021). Increasing Resilience of Production Systems by Integrated Design. Appl. Sci., 11.10.3390/app11188457 ↗
15. Resiliency of Smart Manufacturing Enterprises via Information Integration
16. Big Data-Enabled Large-Scale Group Decision Making for Circular Economy: An Emerging Market Context10.1016/j.techfore.2021.120607 ↗
17. Positioning and Presenting Design Science Research for Maximum Impact10.25300/MISQ/2013/37.2.01 ↗
18. Measuring Information Systems Success: Models, Dimensions, Measures, and Interrelationships10.1057/ejis.2008.15 ↗
19. Transforming Failure into Success through Organisational Learning: An Analysis of a Manufacturing Information System10.1057/palgrave.ejis.3000384 ↗
20. Information Systems Success: The Quest for the Dependent Variable10.1287/isre.3.1.60 ↗
21. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update10.1080/07421222.2003.11045748 ↗
22. Usability: Adoption, Measurement, Value10.1177/0018720819895098 ↗
23. IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use10.1080/10447319509526110 ↗
24. Psychometric Evaluation of the Post-Study System Usability Questionnaire: The PSSUQ10.1177/154193129203601617 ↗
25. Nielsen, J. (1994). Enhancing the Explanatory Power of Usability Heuristics. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Association for Computing Machinery.10.1145/259963.260333 ↗
26. Current Practice in Measuring Usability: Challenges to Usability Studies and Research10.1016/j.ijhcs.2005.06.002 ↗
27. Bjørn, A., and Hauschild, M.Z. (2018). Cradle to Cradle and LCA. Life Cycle Assessment: Theory and Practice, Springer.10.1007/978-3-319-56475-3 ↗
28. Designing Cradle-to-Cradle Products: A Reality Check10.1080/19397030903395166 ↗
29. Toward Improving the Relevance of Information Systems Research to Practice: The Role of Applicability Checks10.2307/25148826 ↗
30. Cloud-Based Information System for Automated Precast Concrete Transportation Planning10.1016/j.autcon.2023.104942 ↗
31. Mokalled, H., Catelli, R., Casola, V., Debertol, D., Meda, E., and Zunino, R. (2019, January 12–14). The Applicability of a SIEM Solution: Requirements and Evaluation. Proceedings of the 2019 IEEE 28th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE), Napoli, Italy.10.1109/WETICE.2019.00036 ↗
32. (2022). Software and Systems Engineering—Software Testing—Part 1: General Concepts (Standard No. ISO/IEC/IEEE ISO/IEC/IEEE 29119-1:2022(En)). Available online: https://www.iso.org/obp/ui/#iso:std:iso-iec-ieee:29119:-1:ed-2:v1:en.
33. Laugwitz, B., Held, T., and Schrepp, M. (2008). Construction and Evaluation of a User Experience Questionnaire. HCI and Usability for Education and Work, Proceedings of the 4th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society, USAB 2008, Graz, Austria, 20–21 November 2008, Springer.10.1007/978-3-540-89350-9_6 ↗
34. Anderson, J.G., and Aydin, C.E. (2005). Qualitative Research Methods for Evaluating Computer Information Systems. Evaluating the Organizational Impact of Healthcare Information Systems, Springer.10.1007/0-387-30329-4 ↗
35. An IoT-Enabled Real-Time Machine Status Monitoring Approach for Cloud Manufacturing10.1016/j.procir.2017.03.349 ↗
36. Lin, S., and Huang, X. (2011, January 21–22). PDCA Cycle and Teaching Reform of Advanced Manufacturing Technology Based on Information. Proceedings of the Advances in Computer Science, Environment, Ecoinformatics, and Education, Wuhan, China.
37. Minimizing Waste Using Lean Manufacturing and ECRS Principle in Indonesian Furniture Industry10.1080/23311916.2019.1567019 ↗
38. Manufacturing Back-Shoring and Direct Brand Creation in the Footwear Industry
39. Jou, Y.T., Silitonga, R.M., Lin, M.C., Sukwadi, R., and Rivaldo, J. (2022). Application of Six Sigma Methodology in an Automotive Manufacturing Company: A Case Study. Sustainability, 14.10.3390/su142114497 ↗
40. Baroni, S., Kalaboukas, K., Aryblia, M., Arampatzis, G., Kallipolitou, D., Psarommatis, F., Zois, G., Kaparis, K., Plitsos, S., and Vatikiotis, S. (2024, August 02). Deliverable 1.3 Sustainability Balanced Scorecard Framework v1, Plooto Project. Available online: https://www.plooto-project.eu/wp-content/uploads/2024/03/Plooto_D1.3_Sustainability-Balanced-Scorecad-Framework-PU.pdf.
41. Past, Present and Future Prospective of Global Carbon Fibre Composite Developments and Applications10.1016/j.compositesb.2022.110463 ↗
42. Denzin, N.K. (2017). The Research Act: A Theoretical Introduction to Sociological Methods, Routledge. [1st ed.].10.4324/9781315134543 ↗
43. Jackson, M., Crouch, S., and Baxter, R. (2011). Software Evaluation: Criteria-Based Assessment 2011, Software Sustainability Institute.
44. SUS-A Quick and Dirty Usability Scale
45. An Empirical Evaluation of the System Usability Scale10.1080/10447310802205776 ↗