A Survey on Bug Deduplication and Triage Methods from Multiple Points of View
Source abstract
Abstract
To address the issue of insufficient testing caused by the continuous reduction of software development cycles, many organizations maintain bug repositories and bug tracking systems to ensure real-time updates of bugs. However, each day, a large number of bugs is discovered and sent to the repository, which imposes a heavy workload on bug fixers. Therefore, effective bug deduplication and triage are of great significance in software development. This paper provides a comprehensive investigation and survey of the recent developments in bug deduplication and triage. The study begins by outlining the roadmap of the existing literature, including the research trends, mathematical models, methods, and commonly used datasets in recent years. Subsequently, the paper summarizes the general process of the methods from two perspectives—runtime information-based and bug report-based perspectives—and provides a detailed overview of the methodologies employed in relevant works. Finally, this paper presents a detailed comparison of the experimental results of various works in terms of usage methods, datasets, accuracy, recall rate, and F1 score. Drawing on key findings, such as the need to improve the accuracy of runtime information collection and refine the description information in bug reports, we propose several potential future research directions in the field, such as stack trace enrichment and the combination of new NLP models.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software Engineering Research · Software System Performance and Reliability · Software Reliability and Analysis Research
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
National Key Laboratory of Science and Technology on Information System Security, Beijing 100085, China
References (100+)
Show source-provided references
1. Anvik, J., Hiew, L., and Murphy, G.C. (2005, January 16–17). Coping with an open bug repository. Proceedings of the 2005 OOPSLA Workshop on Eclipse Technology Exchange, San Diego, CA, USA.10.1145/1117696.1117704 ↗
2. Banerjee, S., Helmick, J., Syed, Z., and Cukic, B. (2015, January 8–10). Eclipse vs. mozilla: A comparison of two large-scale open source problem report repositories. Proceedings of the 2015 IEEE 16th International Symposium on High Assurance Systems Engineering, Daytona Beach Shores, FL, USA.10.1109/HASE.2015.45 ↗
3. Banerjee, S., and Cukic, B. (2015, January 23). On the cost of mining very large open source repositories. Proceedings of the 2015 IEEE/ACM 1st International Workshop on Big Data Software Engineering, Florence, Italy.10.1109/BIGDSE.2015.16 ↗
4. Angell, R., Oztalay, B., and DeOrio, A. (2015, January 3–4). A topological approach to hardware bug triage. Proceedings of the 2015 16th International Workshop on Microprocessor and SOC Test and Verification (MTV), Austin, TX, USA.10.1109/MTV.2015.10 ↗
5. Golagha, M., Lehnhoff, C., Pretschner, A., and Ilmberger, H. (2019, January 15–19). Failure clustering without coverage. Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, Beijing, China.10.1145/3293882.3330561 ↗
6. Mu, D., Wu, Y., Chen, Y., Lin, Z., Yu, C., Xing, X., and Wang, G. (2022, January 24–28). An In-depth Analysis of Duplicated Linux Kernel Bug Reports. Proceedings of the Network and Distributed Systems Security (NDSS) Symposium 2022, San Diego, CA, USA.10.14722/ndss.2022.24159 ↗
7. Systematic Review of Bug Report Processing Techniques to Improve Software Management Performance
8. Jahanshahi, H., Cevik, M., Mousavi, K., and Başar, A. (2022). ADPTriage: Approximate Dynamic Programming for Bug Triage. arXiv.
9. A spatial—Temporal graph neural network framework for automated software bug triaging10.1016/j.knosys.2022.108308 ↗
10. A unified framework for bug report assignment10.1142/S0218194019500256 ↗
11. Improving bug report triage performance using artificial intelligence based document generation model10.1186/s13673-020-00229-7 ↗
12. Neysiani, B.S., and Babamir, S.M. (2020, January 22–23). Automatic duplicate bug report detection using information retrieval-based versus machine learning-based approaches. Proceedings of the 2020 6th International Conference on Web Research (ICWR), Tehran, Iran.10.1109/ICWR49608.2020.9122288 ↗
13. A survey on bug prioritization10.1007/s10462-016-9478-6 ↗
14. A survey on various techniques for bug triage
15. Neysiani, B.S., and Babamir, S.M. (2016, January 1). Methods of feature extraction for detecting the duplicate bug reports in software triage systems. Proceedings of the International Conference on Information Technology, Communications and Telecommunications (IRICT), Tehran, Iran.
16. Survey based classification of bug triage approaches10.11591/APTIKOM.J.CSIT.94 ↗
17. Literature survey on automatic bug triaging using machine learning techniques10.1063/5.0108585 ↗
18. Duplicate Detection Models for Bug Reports of Software Triage Systems: A Survey
19. Automated classification of software issue reports using machine learning techniques: An empirical study10.1007/s11334-017-0294-1 ↗
20. Machine learning or information retrieval techniques for bug triaging: Which is better?
21. Banerjee, S., Cukic, B., and Adjeroh, D. (2012, January 25–27). Automated duplicate bug report classification using subsequence matching. Proceedings of the 2012 IEEE 14th International Symposium on High-Assurance Systems Engineering, Omaha, NE, USA.10.1109/HASE.2012.38 ↗
22. Banerjee, S., Syed, Z., Helmick, J., and Cukic, B. (2013, January 4–7). A fusion approach for classifying duplicate problem reports. Proceedings of the 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE), Pasadena, CA, USA.10.1109/ISSRE.2013.6698920 ↗
23. Prifti, T., Banerjee, S., and Cukic, B. (2011, January 20–21). Detecting bug duplicate reports through local references. Proceedings of the 7th International Conference on Predictive Models in Software Engineering, Banff, AB, Canada.10.1145/2020390.2020398 ↗
24. Fuzzy clustering of crowdsourced test reports for apps10.1145/3106164 ↗
25. Improving automated bug triaging with specialized topic model10.1109/TSE.2016.2576454 ↗
26. Topic modeling and intuitionistic fuzzy set-based approach for efficient software bug triaging10.1007/s10115-022-01735-z ↗
27. Classification and intuitionistic fuzzy set based software bug triaging techniques
28. Jang, J., and Yang, G. (2022). A Bug Triage Technique Using Developer-Based Feature Selection and CNN-LSTM Algorithm. Appl. Sci., 12.10.3390/app12189358 ↗
29. Choquette-Choo, C.A., Sheldon, D., Proppe, J., Alphonso-Gibbs, J., and Gupta, H. (2019, January 16–19). A multi-label, dual-output deep neural network for automated bug triaging. Proceedings of the 2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA), Boca Raton, FL, USA.10.1109/ICMLA.2019.00161 ↗
30. DENATURE: Duplicate detection and type identification in open source bug repositories10.1007/s13198-023-01855-x ↗
31. Does Deep Learning improve the performance of duplicate bug report detection? An empirical study10.1016/j.jss.2023.111607 ↗
32. Dhaliwal, T., Khomh, F., and Zou, Y. (2011, January 25–30). Classifying field crash reports for fixing bugs: A case study of Mozilla Firefox. Proceedings of the 2011 27th IEEE International Conference on Software Maintenance (ICSM), Williamsburg, VA, USA.10.1109/ICSM.2011.6080800 ↗
33. Dang, Y., Wu, R., Zhang, H., Zhang, D., and Nobel, P. (2012, January 2–9). Rebucket: A method for clustering duplicate crash reports based on call stack similarity. Proceedings of the 2012 34th International Conference on Software Engineering (ICSE), Zurich, Switzerland.10.1109/ICSE.2012.6227111 ↗
34. TraceSim: An Alignment Method for Computing Stack Trace Similarity10.1007/s10664-021-10070-w ↗
35. Abaci-finder: Linux kernel crash classification through stack trace similarity learning10.1016/j.jpdc.2022.06.003 ↗
36. Dunn, T., Banerjee, N.K., and Banerjee, S. (2016, January 23–27). GPU acceleration of document similarity measures for automated bug triaging. Proceedings of the 2016 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), Ottawa, ON, Canada.10.1109/ISSREW.2016.27 ↗
37. Wu, R., Zhang, H., Cheung, S.C., and Kim, S. (2014, January 21–25). Crashlocator: Locating crashing faults based on crash stacks. Proceedings of the 2014 International Symposium on Software Testing and Analysis, San Jose, CA, USA.10.1145/2610384.2610386 ↗
38. Koopaei, N.E., and Hamou-Lhadj, A. (2015, January 2–4). CrashAutomata: An approach for the detection of duplicate crash reports based on generalizable automata. Proceedings of the CASCON, Markham, ON, Canada.
39. Sabor, K.K., Hamou-Lhadj, A., and Larsson, A. (2017, January 25–29). Durfex: A feature extraction technique for efficient detection of duplicate bug reports. Proceedings of the 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS), Prague, Czech Republic.10.1109/QRS.2017.35 ↗
40. Tian, Y., Yu, S., Fang, C., and Li, P. (July, January 27). FuRong: Fusing report of automated Android testing on multi-devices. Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings, Seoul, Republic of Korea.
41. Khvorov, A., Vasiliev, R., Chernishev, G., Rodrigues, I.M., Koznov, D., and Povarov, N. (2021, January 17–19). S3M: Siamese stack (trace) similarity measure. Proceedings of the 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), Madrid, Spain.10.1109/MSR52588.2021.00038 ↗
42. Yeh, C.C., Lu, H.L., Lee, Y.H., Chou, W.S., and Huang, S.K. (2017, January 7–10). CRAXTriage: A coverage based triage system. Proceedings of the 2017 IEEE Conference on Dependable and Secure Computing, Taipei, Taiwan.10.1109/DESEC.2017.8073859 ↗
43. Liu, Y. (2022, January 10–14). RESTCluster: Automated Crash Clustering for RESTful API. Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, Rochester, MI, USA.10.1145/3551349.3559511 ↗
44. Deduplication and Exploitability Determination of UAF Vulnerability Samples by Fast Clustering
45. Pham, V.T., Khurana, S., Roy, S., and Roychoudhury, A. (2017, January 22–29). Bucketing failing tests via symbolic analysis. Proceedings of the Fundamental Approaches to Software Engineering: 20th International Conference, FASE 2017, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2017, Uppsala, Sweden. Proceedings 20.
46. Moroo, A., Aizawa, A., and Hamamoto, T. (2017, January 5–7). Reranking-based Crash Report Deduplication. Proceedings of the SEKE, Pittsburgh, PA, USA.10.18293/SEKE2017-135 ↗
47. Cui, W., Peinado, M., Cha, S.K., Fratantonio, Y., and Kemerlis, V.P. (2016, January 14–22). Retracer: Triaging crashes by reverse execution from partial memory dumps. Proceedings of the 38th International Conference on Software Engineering, Austin, TX, USA.10.1145/2884781.2884844 ↗
48. Eom, K.J., Paik, J.Y., Mok, S.K., Jeon, H.G., Cho, E.S., Kim, D.W., and Ryu, J. (2015, January 1–5). Automated crash filtering for arm binary programs. Proceedings of the 2015 IEEE 39th Annual Computer Software and Applications Conference, Taichung, Taiwan.10.1109/COMPSAC.2015.139 ↗
49. Cui, W., Ge, X., Kasikci, B., Niu, B., Sharma, U., Wang, R., and Yun, I. (2018, January 8–10). {REPT}: Reverse debugging of failures in deployed software. Proceedings of the 13th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 18), Carlsbad, CA, USA.
50. Xu, J., Mu, D., Xing, X., Liu, P., Chen, P., and Mao, B. (2017, January 16–18). Postmortem Program Analysis with Hardware-Enhanced Post-Crash Artifacts. Proceedings of the USENIX Security Symposium, Vancouver, BC, Canada.
51. Pomp++: Facilitating postmortem program diagnosis with value-set analysis
52. Jiang, Z., Jiang, X., Hazimeh, A., Tang, C., Zhang, C., and Payer, M. (2021, January 15–19). Igor: Crash Deduplication Through Root-Cause Clustering. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, Virtual Event, Republic of Korea.10.1145/3460120.3485364 ↗
53. van Tonder, R., Kotheimer, J., and Le Goues, C. (2018, January 3–7). Semantic crash bucketing. Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, Montpellier, France.10.1145/3238147.3238200 ↗
54. Zhang, X., Chen, J., Feng, C., Li, R., Diao, W., Zhang, K., Lei, J., and Tang, C. (2022, January 21–29). DeFault: Mutual information-based crash triage for massive crashes. Proceedings of the 44th International Conference on Software Engineering, Pittsburgh, PA, USA.10.1145/3510003.3512760 ↗
55. Kallingal Joshy, A., and Le, W. (2022, January 10–14). FuzzerAid: Grouping Fuzzed Crashes Based On Fault Signatures. Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, Rochester, MI, USA.10.1145/3551349.3556959 ↗
56. Alawneh, A., Alazzam, I.M., and Shatnawi, K. (2022). Locating Source Code Bugs in Software Information Systems Using Information Retrieval Techniques. Big Data Cogn. Comput., 6.10.3390/bdcc6040156 ↗
57. Krasniqi, R., and Do, H. (2022, January 13–15). Automatically Capturing Quality-Related Concerns in Bug Report Descriptions for Efficient Bug Triaging. Proceedings of the International Conference on Evaluation and Assessment in Software Engineering 2022, Gothenburg, Sweden.10.1145/3530019.3530021 ↗
58. Lee, C.Y., Hu, D.D., Feng, Z.Y., and Yang, C.Z. (2015, January 12–16). Mining temporal information to improve duplication detection on bug reports. Proceedings of the 2015 IIAI 4th International Congress on Advanced Applied Informatics, Okayama, Japan.10.1109/IIAI-AAI.2015.180 ↗
59. Improving bug management using correlations in crash reports10.1007/s10664-014-9333-9 ↗
60. Revisiting the performance evaluation of automated approaches for the retrieval of duplicate issue reports10.1109/TSE.2017.2755005 ↗
61. Automated triaging of very large bug repositories10.1016/j.infsof.2016.09.006 ↗
62. Savidov, G., and Fedotov, A. (2021, January 2–3). Casr-Cluster: Crash clustering for Linux applications. Proceedings of the 2021 Ivannikov Ispras Open Conference (ISPRAS).10.1109/ISPRAS53967.2021.00012 ↗
63. Budhiraja, A., Reddy, R., and Shrivastava, M. (June, January 27). Lwe: Lda refined word embeddings for duplicate bug report detection. Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings, Gothenburg, Sweden.
64. Chaparro, O., Florez, J.M., Singh, U., and Marcus, A. (2019, January 24–27). Reformulating queries for duplicate bug report detection. Proceedings of the 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER), Hangzhou, China.10.1109/SANER.2019.8667985 ↗
65. Karasov, N., Khvorov, A., Vasiliev, R., Golubev, Y., and Bryksin, T. (2022). Aggregation of Stack Trace Similarities for Crash Report Deduplication. arXiv.
66. James, K., Du, Y., Das, S., and Monrose, F. (2022, January 5–9). Separating the Wheat from the Chaff: Using Indexing and Sub-Sequence Mining Techniques to Identify Related Crashes During Bug Triage. Proceedings of the 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS), Guangzhou, China.10.1109/QRS57517.2022.00014 ↗
67. Yang, H., Xu, Y., Li, Y., and Choi, H.D. (2020, January 12–15). K-Detector: Identifying Duplicate Crash Failures in Large-Scale Software Delivery. Proceedings of the 2020 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), Coimbra, Portugal.10.1109/ISSREW51248.2020.00028 ↗
68. Park, J.w., Lee, M.W., Kim, J., Hwang, S.w., and Kim, S. (2011, January 7–11). Costriage: A cost-aware triage algorithm for bug reporting systems. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v25i1.7839 ↗
69. A contextual approach towards more accurate duplicate bug report detection and ranking10.1007/s10664-015-9387-3 ↗
70. Badashian, A.S. (2016, January 14–22). Realistic bug triaging. Proceedings of the 38th International Conference on Software Engineering Companion, Austin, TX, USA.10.1145/2889160.2889268 ↗
71. Guiding bug triage through developer analysis in bug reports10.1142/S0218194016500170 ↗
72. Goyal, A. (2017, January 20–28). Effective Bug Triage for Non-Reproducible Bugs. Proceedings of the 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C), Buenos Aries, Argentina.10.1109/ICSE-C.2017.41 ↗
73. Zhang, W., Cui, Y., and Yoshida, T. (2017). En-lda: An novel approach to automatic bug report assignment with entropy optimized latent dirichlet allocation. Entropy, 19.10.3390/e19050173 ↗
74. Preventing duplicate bug reports by continuously querying bug reports10.1007/s10664-018-9643-4 ↗
75. Ranking of software developers based on expertise score for bug triaging10.1016/j.infsof.2019.03.014 ↗
76. Automatic bug triage in software systems using graph neighborhood relations for feature augmentation10.1109/TCSS.2020.3017501 ↗
77. Nath, V., Sheldon, D., and Alphonso-Gibbs, J. (2021, January 13–15). Principal Component Analysis and Entropy-based Selection for the Improvement of Bug Triage. Proceedings of the 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), Virtually Online.10.1109/ICMLA52953.2021.00090 ↗
78. Panda, R.R., and Nagwani, N.K. (2022). Proceedings of the Third Doctoral Symposium on Computational Intelligence: DoSCI 2022, Springer.
79. Wu, X., Shan, W., Zheng, W., Chen, Z., Ren, T., and Sun, X. (2023, January 15–16). An Intelligent Duplicate Bug Report Detection Method Based on Technical Term Extraction. Proceedings of the 2023 IEEE/ACM International Conference on Automation of Software Test (AST), Melbourne, Australia.10.1109/AST58925.2023.00005 ↗
80. An HMM-based approach for automatic detection and classification of duplicate bug reports10.1016/j.infsof.2019.05.007 ↗
81. Rodrigues, I.M., Aloise, D., Fernandes, E.R., and Dagenais, M. (2020, January 29–30). A soft alignment model for bug deduplication. Proceedings of the 17th International Conference on Mining Software Repositories, Virtual Online.10.1145/3379597.3387470 ↗
82. He, J., Xu, L., Yan, M., Xia, X., and Lei, Y. (2020, January 13–15). Duplicate bug report detection using dual-channel convolutional neural networks. Proceedings of the 28th International Conference on Program Comprehension, Virtual Online.10.1145/3387904.3389263 ↗
83. Detecting duplicate bug reports with software engineering domain knowledge10.1002/smr.1821 ↗
84. Dedík, V., and Rossi, B. (September, January 31). Automated bug triaging in an industrial context. Proceedings of the 2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Limassol, Cyprus.10.1109/SEAA.2016.20 ↗
85. Enhancements for duplication detection in bug reports with manifold correlation features10.1016/j.jss.2016.02.022 ↗
86. Lee, S.R., Heo, M.J., Lee, C.G., Kim, M., and Jeong, G. (2017, January 4–8). Applying deep learning based automatic bug triager to industrial projects. Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, Paderborn, Germany.10.1145/3106237.3117776 ↗
87. Xuan, J., Jiang, H., Ren, Z., Yan, J., and Luo, Z. (2017). Automatic bug triage using semi-supervised text classification. arXiv.
88. DeepTriage:An Automatic Triage Method for Software Bugs Using Deep Learning
89. Chaparro, O. (2017, January 20–28). Improving bug reporting, duplicate detection, and localization. Proceedings of the 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C), Buenos Aires, Argentina.10.1109/ICSE-C.2017.27 ↗
90. Xi, S., Yao, Y., Xiao, X., Xu, F., and Lu, J. (2018, January 16). An effective approach for routing the bug reports to the right fixers. Proceedings of the 10th Asia-Pacific Symposium on Internetware, Beijing, China.10.1145/3275219.3275228 ↗
91. Xie, Q., Wen, Z., Zhu, J., Gao, C., and Zheng, Z. (2018, January 4–7). Detecting duplicate bug reports with convolutional neural networks. Proceedings of the 2018 25th Asia-Pacific Software Engineering Conference (APSEC), Nara, Japan.10.1109/APSEC.2018.00056 ↗
92. Alenezi, M., Banitaan, S., and Zarour, M. (2018). Using categorical features in mining bug tracking systems to assign bug reports. arXiv.10.5121/ijsea.2018.9203 ↗
93. Budhiraja, A., Dutta, K., Reddy, R., and Shrivastava, M. (June, January 27). DWEN: Deep word embedding network for duplicate bug report detection in software repositories. Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings, Gothenburg, Sweden.
94. Kukkar, A., Mohana, R., Nayyar, A., Kim, J., Kang, B.G., and Chilamkurti, N. (2019). A novel deep-learning-based bug severity classification technique using convolutional neural networks and random forest with boosting. Sensors, 19.10.3390/s19132964 ↗
95. Bug triaging based on tossing sequence modeling10.1007/s11390-019-1953-5 ↗
96. Mani, S., Sankaran, A., and Aralikatte, R. (2019, January 3–5). Deeptriage: Exploring the effectiveness of deep learning for bug triaging. Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, Kolkata, India.10.1145/3297001.3297023 ↗
97. Not all bugs are the same: Understanding, characterizing, and classifying bug types10.1016/j.jss.2019.03.002 ↗
98. Poddar, L., Neves, L., Brendel, W., Marujo, L., Tulyakov, S., and Karuturi, P. (2019). Train one get one free: Partially supervised neural network for bug report duplicate detection and clustering. arXiv.10.18653/v1/N19-2020 ↗
99. Sarkar, A., Rigby, P.C., and Bartalos, B. (October, January 29). Improving bug triaging with high confidence predictions at ericsson. Proceedings of the 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME), Cleveland, OH, USA.10.1109/ICSME.2019.00018 ↗
100. Pahins, C.A.D.L., D’Morison, F., Rocha, T.M., Almeida, L.M., Batista, A.F., and Souza, D.F. (2019, January 16–19). T-REC: Towards accurate bug triage for technical groups. Proceedings of the 2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA), Boca Raton, FL, USA.