The MADAG Strategy for Fault Location Techniques
Source abstract
Abstract
Spectrum-based fault localization (SBFL), which utilizes spectrum information of test cases to calculate the suspiciousness of each statement in a program, can reduce developers’ effort. However, applying redundant test cases from a test suite to fault localization incurs a heavy burden, especially in a restricted resource environment, and it is expensive and infeasible to inspect the results of each test input. Prioritizing/selecting appropriate test cases is important to enable the practical application of the SBFL technique. In addition, we must ensure that applying the selected tests to SBFL can achieve approximately the effectiveness of fault localization with whole tests. This paper presents a test case prioritization/selection strategy, namely the Minimal Aggregate of the Diversity of All Groups (MADAG). The MADAG strategy prioritizes/selects test cases using information on the diversity of the execution trace of each test case. We implemented and applied the MADAG strategy to 233 faulty versions of the Siemens and UNIX programs from the Software-artifact Infrastructure Repository. The experiments show that (1) the MADAG strategy uses only 8.99 and 14.27 test cases, with an average of 18, from the Siemens and UNIX test suites, respectively, and the SBFL technique has approximate effectiveness for fault localization on all test cases and outperforms the previous best test case prioritization method; (2) we verify that applying whole tests from the test suite may not achieve the better effectiveness in fault localization compared with the tests selected by MADAG strategy.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software Testing and Debugging Techniques · Software Engineering Research · Software Reliability and Analysis Research
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Ausenior Information Co., Ltd., Taipei City 11493, Taiwan
Department of Applied Informatics, Fo-Guang University, Yilan County 26247, Taiwan
Funding
National Science and Technology Council (NSTC) of Taiwan
References (46)
Show source-provided references
1. Abreu, R., Zoeteweij, P., and van Gemund, A.J.C. (2009, January 16–20). Spectrum-Based Multiple Fault Localization. Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE), Auckland, New Zealand.10.1109/ASE.2009.25 ↗
2. A practical evaluation of spectrum-based fault localization10.1016/j.jss.2009.06.035 ↗
3. Spectrum-based fault localization techniques application on multiple-fault programs: A review
4. Historical spectrum based fault localization10.1109/TSE.2019.2948158 ↗
5. The DStar method for effective software fault localization10.1109/TR.2013.2285319 ↗
6. Godefroid, P., Klarlund, N., and Sen, K. (2005, January 12–15). Dart: Directed automated random testing. Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, Chicago, IL, USA.10.1145/1065010.1065036 ↗
7. Ma, E., Fu, X., and Wang, X. (2022). Scalable path search for automated test case generation. Electronics, 11.10.3390/electronics11050727 ↗
8. Sen, K., Marinov, D., and Agha, G. (2005, January 5–9). Cute: A concolic unit testing engine for c. Proceedings of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), Lisbon, Portugal.10.1145/1095430.1081750 ↗
9. Yu, Y., Jones, J.A., and Harrold, M.J. (2008, January 10–18). An empirical study of the effects of test-suite reduction on fault localization. Proceedings of the 30th International Conference on Software Engineering (ICSE), Leipzig, Germany.
10. Test input reduction for result inspection to facilitate fault localization10.1007/s10515-009-0056-x ↗
11. Prioritizing tests for software fault diagnosis10.1002/spe.1065 ↗
12. Diversity maximization speedup for localizing faults in single-fault and multi-fault programs10.1007/s10515-014-0165-z ↗
13. Theoretical Analysis and Empirical Study on the Impact of Coincidental Correct Test Cases in Multiple Fault Localization10.1109/TR.2022.3165126 ↗
14. Renieris, M., and Reiss, S. (2003, January 6–10). Fault localization with nearest neighbor queries. Proceedings of the IEEE/ACM 18th International Conference on Automated Software Engineering (ASE), Montreal, QC, Canada.
15. Wong, W.E., Horgan, J.R., London, S., and Agrawal, H. (1997, January 2–5). A study of effective regression testing in practice. Proceedings of the 8th International Symposium on Software Reliability Engineering (ISSRE), Albuquerque, NM, USA.
16. Jain, A.K., and Dubes, R.C. (1988). Algorithms for Clustering Data, Prentice-Hall, Inc.
17. Chen, M.Y., Kiciman, E., Fratkin, E., Fox, A., and Brewer, E. (2002, January 23–26). Pinpoint: Problem determination in large, dynamic Internet services. Proceedings of the International Conference on Dependable Systems and Networks (DSN), Washington, DC, USA.
18. Jones, J.A., and Harrold, M.J. (2005, January 7–11). Empirical Evaluation of the Tarantula Automatic Fault-Localization Technique. Proceedings of the IEEE/ACM 20th International Conference on Automated Software Engineering (ASE), Long Beach, CA, USA.10.1145/1101908.1101949 ↗
19. A systematic literature review of test case prioritization using genetic algorithms10.1109/ACCESS.2019.2938260 ↗
20. Prioritizing test cases for regression testing10.1109/32.962562 ↗
21. Jiang, B., Zhang, Z., Chan, W.K., and Tse, T.H. (2009, January 16–20). Adaptive Random Test Case Prioritization. Proceedings of the IEEE/ACM 24th International Conference on Automated Software Engineering (ASE), Auckland, New Zealand.10.1109/ASE.2009.77 ↗
22. Arcuri, A., and Briand, L.C. (2011, January 17–21). Adaptive random testing: An illusion of effectiveness?. Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), Toronto, ON, Canana.10.1145/2001420.2001452 ↗
23. PIE: A dynamic failure-based techniques10.1109/32.153381 ↗
24. Gonzalez-Sanchez, A., Abreu, R., Gross, H.-G., and van Gemund, A.J.C. (2011, January 6–10). Prioritizing tests for fault localization through ambiguity group reduction. Proceedings of the IEEE/ACM 26th International Conference on Automated Software Engineering (ASE), Lawrence, KS, USA.10.1109/ASE.2011.6100153 ↗
25. Multiple fault localization of software programs: A systematic literature review10.1016/j.infsof.2020.106312 ↗
26. How well does test case prioritization integrate with statistical fault localization?10.1016/j.infsof.2012.01.006 ↗
27. A family of code coverage-based heuristics for effective fault localization10.1016/j.jss.2009.09.037 ↗
28. Santelices, R., Jones, J.A., Yu, Y., and Harrold, M.J. (2009, January 16–24). Lightweight Fault-localization Using Multiple Coverage Types. Proceedings of the 31th International Conference on Software Engineering (ICSE), Vancouver, BC, Canada.10.1109/ICSE.2009.5070508 ↗
29. Jeffrey, D., Gupta, N., and Gupta, R. (2008, January 20–24). Fault localization using value replacement. Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), Seattle, WA, USA.10.1145/1390630.1390652 ↗
30. (2022, November 03). GCC, the GNU Compiler Collection. Available online: https://gcc.gnu.org/.
31. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact10.1007/s10664-005-3861-2 ↗
32. Regression testing minimization, selection and prioritization: A survey10.1002/stvr.430 ↗
33. Jones, J.A., Harrold, M.J., and Stasko, J. (2002, January 19–25). Visualization of test information to assist fault localization. Proceedings of the 24th International Conference on Software Engineering (ICSE), Orlando, FL, USA.10.1145/581339.581397 ↗
34. Fault Localization Based Only on Failed Runs10.1109/MC.2012.185 ↗
35. Analytic hierarchy process-based regression test case prioritization technique enhancing the fault detection rate10.1007/s00500-022-07174-w ↗
36. Test case prioritization: A family of empirical studies10.1109/32.988497 ↗
37. Search algorithms for regression test case prioritization10.1109/TSE.2007.38 ↗
38. Baudry, B., Fleurey, F., and Traon, Y.L. (2006, January 20–28). Improving test suites for efficient fault localization. Proceedings of the ICSE, Shanghai, China.10.1145/1134285.1134299 ↗
39. Liu, C., Yan, X., Fei, L., Han, J., and Midkiff, S.P. (9, January 5). SOBER: Statistical model-based bug localization. Proceedings of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Lisbon, Portugal.
40. Feature-FL: Feature-Based Fault Localization10.1109/TR.2022.3140453 ↗
41. Ramamohanarao, K. A model for spectra-based software diagnosis10.1145/2000791.2000795 ↗
42. A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization10.1145/2522920.2522924 ↗
43. Extended comprehensive study of association measures for fault localization10.1002/smr.1616 ↗
44. Cleve, H., and Zeller, A. (2005, January 15–21). Locating causes of program failure. Proceedings of the 27th International Conference on Software Engineering (ICSE), St. Louis, MO, USA.10.1145/1062455.1062522 ↗
45. Wang, T., and Roychoudhury, A. (2005, January 7–11). Automated path generation for software fault localization. Proceedings of the IEEE/ACM 20th International Conference on Automated Software Engineering (ASE), Long Beach, CA, USA.10.1145/1101908.1101966 ↗
46. Campos, J., Abreu, R., Fraser, G., and d’Amorim, M. (2013, January 11–15). Entropy-based test generation for improved fault localization. Proceedings of the IEEE/ACM 28th International Conference on Automated Software Engineering (ASE), Silicon Valley, CA, USA.10.1109/ASE.2013.6693085 ↗