Pattern-Based Test Suite Reduction Method for Smart Contracts
Source abstract
Abstract
Testing in the software development process is one of the crucial disciplines. The immutability of deployed smart contracts makes tests particularly important in their design and implementation. The objective is to minimize the costs of smart contract testing by reducing the number of test cases. The article presents the test suite reduction method for smart contracts, which exploits the knowledge of their design manner. In particular, the design of a mechanism for evaluating the list of verification rules controlling the execution of transactions was used. The proposed method defines a minimum set of test cases for smart contracts designed according to a specific design pattern. The test case reduction level was shown using analytical formulas enclosed in the paper. Compared to full coverage, the method reduces over 96% of test cases for a smart contract with at least eight verification rules. The performance tests were conducted and the analysis of the results was enclosed as well. Both the smart contract evaluation time and the test suite execution time are less than 0.01 of a millisecond. Besides, the method reduces the number of test data for smart contracts by over 79%. Therefore, the method can be especially useful in continuous software integration and delivery.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software Testing and Debugging Techniques · Advanced Malware Detection Techniques · Software System Performance and Reliability
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Computer Science, Gdynia Maritime University, Morska 81-87, 81-225 Gdynia, Poland
References (48)
Show source-provided references
1. (2017). IEC/IEEE International Standard-Systems and Software Engineering–Vocabulary (Standard No. ISO/IEC/IEEE 24765: 2017 (E)).
2. An experimental study of four typical test suite reduction techniques10.1016/j.infsof.2007.06.003 ↗
3. A Systematic Review on Test Suite Reduction: Approaches, Experiment’s Quality Evaluation, and Guidelines10.1109/ACCESS.2018.2809600 ↗
4. Adequate vs. inadequate test suite reduction approaches10.1016/j.infsof.2019.106224 ↗
5. Assessing test artifact quality—A tertiary study10.1016/j.infsof.2021.106620 ↗
6. Removing redundant refusals: Minimal complete test suites for failure trace semantics10.1016/j.ic.2023.105009 ↗
7. Górski, T. (2022). The k + 1 Symmetric Test Pattern for Smart Contracts. Symmetry, 14.10.3390/sym14081686 ↗
8. Górski, T. (2024). Smart Contract Design Pattern for Processing Logically Coherent Transaction Types. Appl. Sci., 14.10.3390/app14062224 ↗
9. (2023). Systems and Software Engineering—Systems and Software Quality Requirements and Evaluation (SQuaRE)—Product Quality Model (Standard No. ISO/IEC 25010:2023). Available online: https://www.iso.org/standard/78176.html.
10. A General Testability Theory: Classes, Properties, Complexity, and Testing Reductions10.1109/TSE.2014.2331690 ↗
11. On the divide-and-conquer approach towards test suite reduction10.1016/S0020-0255(03)00060-4 ↗
12. Alsharif, A., Kapfhammer, G.M., and McMinn, P. (2020, January 24–28). STICCER: Fast and Effective Database Test Suite Reduction Through Merging of Similar Test Cases. Proceedings of the 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), Porto, Portugal.10.1109/ICST46399.2020.00031 ↗
13. Candidate test set reduction for adaptive random testing: An overheads reduction technique10.1016/j.scico.2021.102730 ↗
14. Pan, R., Ghaleb, T.A., and Briand, L. (2023, January 14–20). ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolutionary Search. Proceedings of the 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), Melbourne, Australia.10.1109/ICSE48619.2023.00146 ↗
15. Koitz-Hristov, R., Stracke, L., and Wotawa, F. (2022, January 17–18). Checked Coverage for Test Suite Reduction—Is It Worth the Effort?. Proceedings of the 2022 IEEE/ACM International Conference on Automation of Software Test (AST), Pittsburgh, PA, USA.10.1145/3524481.3527216 ↗
16. Statement frequency coverage: A code coverage criterion for assessing test suite effectiveness10.1016/j.infsof.2020.106426 ↗
17. Test Suite Reduction via Evolutionary Clustering10.1109/ACCESS.2021.3058301 ↗
18. Gharachorlu, G., and Sumner, N. (2021, January 17–19). Leveraging Models to Reduce Test Cases in Software Repositories. Proceedings of the 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), Madrid, Spain.10.1109/MSR52588.2021.00035 ↗
19. Combining Code and Requirements Coverage with Execution Cost for Test Suite Reduction10.1109/TSE.2017.2777831 ↗
20. IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization10.1109/ACCESS.2020.3004145 ↗
21. Multi-objective test suite optimization for detection and localization of software faults
22. Test suite reduction methods that decrease regression testing costs by identifying irreplaceable tests10.1016/j.infsof.2014.04.013 ↗
23. Empirically evaluating Greedy-based test suite reduction methods at different levels of test suite complexity10.1016/j.scico.2017.05.004 ↗
24. Evaluation and analysis of incorporating Fuzzy Expert System approach into test suite reduction10.1016/j.infsof.2016.07.005 ↗
25. A methodology for controlling the size of a test suite10.1145/152388.152391 ↗
26. A new heuristic for test suite reduction10.1016/S0950-5849(98)00050-0 ↗
27. A Greedy Heuristic for the Set-Covering Problem10.1287/moor.4.3.233 ↗
28. Gan, N., Wang, H., Zhao, Y., and Yang, F. (2021, January 15–17). Test case set reduction method based on gre and ant colony algorithm. Proceedings of the 2021 IEEE International Conference on Consumer Electronics and Computer Engineering (ICCECE), Guangzhou, China.10.1109/ICCECE51280.2021.9342065 ↗
29. An effective formulation of the multi-criteria test suite minimization problem10.1016/j.jss.2020.110632 ↗
30. An empirical comparison of four Java-based regression test selection techniques10.1016/j.jss.2021.111174 ↗
31. d’Aragona, D.A., Pecorelli, F., Romano, S., Scanniello, G., Baldassarre, M.T., Janes, A., and Lenarduzzi, V. (2022, January 3–7). CATTO: Just-in-time Test Case Selection and Execution. Proceedings of the 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME), Limassol, Cyprus.10.1109/ICSME55016.2022.00059 ↗
32. Using mutual information to test from Finite State Machines: Test suite selection10.1016/j.infsof.2020.106498 ↗
33. Exhaustive property oriented model-based testing with symbolic finite state machines10.1016/j.scico.2023.103005 ↗
34. Model-Based Software Design and Testing in Blockchain Smart Contracts: A Systematic Literature Review10.1109/ACCESS.2020.3021502 ↗
35. Ji, S., Dong, J., Qiu, J., Gu, B., Wang, Y., and Wang, T. (2021, January 6–10). Increasing Fuzz Testing Coverage for Smart Contracts with Dynamic Taint Analysis. Proceedings of the 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS), Hainan, China.10.1109/QRS54544.2021.00035 ↗
36. Effuzz: Efficient fuzzing by directed search for smart contracts10.1016/j.infsof.2023.107213 ↗
37. CrossFuzz: Cross-Contract Fuzzing for Smart Contract Vulnerability Detection10.1016/j.scico.2023.103076 ↗
38. Du, J., Huang, S., Wang, X., Zheng, C., and Sun, J. (2022, January 5–9). Test Case Generation for Ethereum Smart Contract based on Data Dependency Analysis of State Variable. Proceedings of the 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS), Guangzhou, China.10.1109/QRS57517.2022.00077 ↗
39. Fooladgar, M., Arefzadeh, A., and Faghih, F. (2021, January 28–29). TestSmart: A Tool for Automated Generation of Effective Test Cases for Smart Contracts. Proceedings of the 2021 11th International Conference on Computer Engineering and Knowledge (ICCKE), Mashhad, Iran.10.1109/ICCKE54056.2021.9721448 ↗
40. SolAR: Automated test-suite generation for solidity smart contracts10.1016/j.scico.2023.103036 ↗
41. Wang, X., Xie, Z., He, J., Zhao, G., and Nie, R. (2019, January 17–19). Basis Path Coverage Criteria for Smart Contract Application Testing. Proceedings of the 2019 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), Guilin, China.10.1109/CyberC.2019.00016 ↗
42. Olsthoorn, M., Stallenberg, D., Van Deursen, A., and Panichella, A. (2022, January 21–29). SynTest-Solidity: Automated Test Case Generation and Fuzzing for Smart Contracts. Proceedings of the 2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), Pittsburgh, PA, USA.10.1145/3510454.3516869 ↗
43. A Improving Coverage and Vulnerability Detection in Smart Contract Testing Using Self-Adaptive Learning GA10.1080/03772063.2023.2280672 ↗
44. TaintGuard: Preventing implicit privilege leakage in smart contract based on taint tracking at abstract syntax tree level10.1016/j.sysarc.2023.102925 ↗
45. AdapT: A reusable package for implementing smart contracts that process transactions of congruous types10.1016/j.simpa.2024.100694 ↗
46. SmarTS: A Java package for smart contract test suite generation and execution10.1016/j.softx.2024.101698 ↗
47. Verification of Architectural Views Model 1+5 Applicability10.1007/978-3-030-45093-9_60 ↗
48. Górski, T. (2021). The 1+5 Architectural Views Model in Designing Blockchain and IT System Integration Solutions. Symmetry, 13.10.3390/sym13112000 ↗