DAG-Guided Active Fuzzing: A Deterministic Approach to Detecting Race Conditions in Distributed Cloud Systems
Source abstract
Abstract
The rapid expansion of distributed cloud platforms introduces critical security challenges, specifically non-deterministic race conditions like Time-of-Check to Time-of-Use (TOCTOU) vulnerabilities. Traditional passive detection methods often fail to identify these transient “Heisenbugs” due to the asynchronous nature of multi-threaded control planes. To address this, we propose a novel DAG-Guided Active Fuzzing framework. Our approach constructs a Directed Acyclic Graph (DAG) to map causal dependencies of API operations and implements deterministic proactive scheduling. By injecting microsecond-level delays into identified race windows, the system enforces adversarial interleavings to expose hidden order and atomicity violations. Validated on 32 verified vulnerabilities across six distributed systems (including Hadoop and OpenStack), our method achieves an overall Recall (Detection Rate) of 68.8% across the entire dataset and a peak Precision of 92% in reproducibility tests, significantly outperforming random fuzzing baselines (p<0.01). Furthermore, the framework maintains a low runtime overhead of 11.5%. These findings demonstrate a favorable trade-off between detection depth and system efficiency, establishing the approach as a robust toolchain for transforming theoretical concurrency risks into reproducible security findings in large-scale cloud infrastructure.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software System Performance and Reliability · Security and Verification in Computing · Cloud Data Security Solutions
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Cyber Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China
Funding
National Natural Science Foundation of China
References (40)
Show source-provided references
1. Cowan, C., Beattie, S., Wright, C., and Kroah-Hartman, G. (2001, January 13–17). {RaceGuard}: Kernel Protection From Temporary File Race Vulnerabilities. Proceedings of the 10th USENIX Security Symposium (USENIX Security 01), Washington, DC, USA.
2. Race against time: Investigating the factors that influence web race condition exploits10.1016/j.cose.2025.104740 ↗
3. Cotroneo, D., De Simone, L., Liguori, P., Natella, R., and Bidokhti, N. (2019, January 26–30). How bad can a bug get? An empirical analysis of software failures in the openstack cloud computing platform. Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Tallinn, Estonia.10.1145/3338906.3338916 ↗
4. Musavi, P., Adams, B., and Khomh, F. (2016, January 23–27). Experience report: An empirical study of API failures in OpenStack cloud environments. Proceedings of the 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), Ottawa, ON, Canada.10.1109/ISSRE.2016.42 ↗
5. Musuvathi, M., Qadeer, S., Ball, T., Basler, G., Nainar, P.A., and Neamtiu, I. (2008, January 8–10). Finding and Reproducing Heisenbugs in Concurrent Programs. Proceedings of the 8th USENIX Symposium on Operating Systems Design and Implementation, San Diego, CA, USA.
6. Fuzzing: State of the art10.1109/TR.2018.2834476 ↗
7. Full-stack vulnerability analysis of the cloud-native platform10.1016/j.cose.2023.103173 ↗
8. Leesatapornwongsa, T., Lukman, J.F., Lu, S., and Gunawi, H.S. (2016, January 2–6). TaxDC: A taxonomy of non-deterministic concurrency bugs in datacenter distributed systems. Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems, Atlanta, GA, USA.10.1145/2954680.2872374 ↗
9. Lu, J., Li, F., Li, L., and Feng, X. (2018, January 4–9). Cloudraid: Hunting concurrency bugs in the cloud via log-mining. Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Lake Buena Vista, FL, USA.10.1145/3236024.3236071 ↗
10. Xu, L., Huang, J., Hong, S., Zhang, J., and Gu, G. (2017, January 16–18). Attacking the brain: Races in the SDN control plane. Proceedings of the 26th USENIX Security Symposium (USENIX Security), Vancouver, BC, Canada.
11. Tang, H., Wu, G., Wei, J., and Zhong, H. (2016, January 3–7). Generating test cases to expose concurrency bugs in android applications. Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, Singapore.10.1145/2970276.2970320 ↗
12. Scheduling multi-tenant cloud workflow tasks with resource reliability10.1007/s11432-020-3295-2 ↗
13. Gu, Y., and Mellor-Crummey, J. (2018, January 11–16). Dynamic data race detection for OpenMP programs. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, Dallas, TX, USA.10.1109/SC.2018.00064 ↗
14. Fuzzing Code with AFL
15. Fioraldi, A., Maier, D.C., Zhang, D., and Balzarotti, D. (2022, January 7–11). Libafl: A framework to build modular and reusable fuzzers. Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, Los Angeles, CA, USA.10.1145/3548606.3560602 ↗
16. Hong, X.J., Yang, H.S., and Kim, Y.H. (2018, January 17–19). Performance analysis of RESTful API and RabbitMQ for microservice web application. Proceedings of the 2018 International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Republic of Korea.10.1109/ICTC.2018.8539409 ↗
17. Cloud security based attack detection using transductive learning integrated with Hidden Markov Model10.1016/j.patrec.2022.02.012 ↗
18. Serebryany, K., and Iskhodzhanov, T. (2009, January 12). ThreadSanitizer: Data race detection in practice. Proceedings of the Workshop on Binary Instrumentation and Applications, New York, NY, USA.10.1145/1791194.1791203 ↗
19. Cai, Y., and Chan, W.K. (2012, January 2–9). MagicFuzzer: Scalable deadlock detection for large-scale applications. Proceedings of the 2012 34th International Conference on Software Engineering (ICSE), Zurich, Switzerland.10.1109/ICSE.2012.6227156 ↗
20. Lin, Y., and Kulkarni, S.S. (2014, January 21–25). Automatic repair for multi-threaded programs with deadlock/livelock using maximum satisfiability. Proceedings of the 2014 International Symposium on Software Testing and Analysis, San Jose, CA, USA.10.1145/2610384.2610398 ↗
21. Han, X., Schooley, R., Mackenzie, D., David, O., and Lloyd, W.J. (2020, January 21–24). Characterizing public cloud resource contention to support virtual machine co-residency prediction. Proceedings of the 2020 IEEE International Conference on Cloud Engineering (IC2E), Sydney, Australia.10.1109/IC2E48712.2020.00024 ↗
22. Data sequence signal manipulation in multipath TCP (MPTCP): The vulnerability, attack and its detection10.1016/j.cose.2021.102180 ↗
23. OpenAPI framework for the Web of Things10.1016/j.iot.2022.100675 ↗
24. Atlidakis, V., Godefroid, P., and Polishchuk, M. (2019, January 25–31). Restler: Stateful rest api fuzzing. Proceedings of the 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), Montreal, QC, Canada.10.1109/ICSE.2019.00083 ↗
25. Atlidakis, V., Geambasu, R., Godefroid, P., Polishchuk, M., and Ray, B. (2020). Pythia: Grammar-based fuzzing of rest apis with coverage-guided feedback and learning-based mutations. arXiv.
26. Godefroid, P., Lehmann, D., and Polishchuk, M. (2020, January 18–22). Differential regression testing for REST APIs. Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, Virtual.10.1145/3395363.3397374 ↗
27. Du, W., Li, J., Wang, Y., Chen, L., Zhao, R., Zhu, J., Han, Z., Wang, Y., and Xue, Z. (2024, January 14–16). Vulnerability-oriented testing for restful apis. Proceedings of the 33rd USENIX Security Symposium (USENIX Security 24), Philadelphia, PA, USA.
28. Lyu, C., Xu, J., Ji, S., Zhang, X., Wang, Q., Zhao, B., Pan, G., Cao, W., Chen, P., and Beyah, R. (2023, January 9–11). MINER: A Hybrid Data-Driven Approach for REST API Fuzzing. Proceedings of the 32nd USENIX Security Symposium (USENIX Security), Anaheim, CA, USA.
29. RESTLess: Enhancing State-of-the-Art REST API Fuzzing With LLMs in Cloud Service Computing10.1109/TSC.2024.3489441 ↗
30. Böhme, M., Pham, V.T., and Roychoudhury, A. (2016, January 24–28). Coverage-based greybox fuzzing as markov chain. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria.10.1145/2976749.2978428 ↗
31. Lee, G., Shim, W., and Lee, B. (2021, January 11–13). Constraint-guided directed greybox fuzzing. Proceedings of the 30th USENIX Security Symposium (USENIX Security 21), Virtual.
32. Padhye, R., Lemieux, C., Sen, K., Papadakis, M., and Le Traon, Y. (2019, January 15–19). Semantic fuzzing with zest. Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, Beijing, China.10.1145/3293882.3330576 ↗
33. Lee, S., Han, H., Cha, S.K., and Son, S. (2020, January 12–14). Montage: A neural network language Model-Guided JavaScript engine fuzzer. Proceedings of the 29th USENIX Security Symposium (USENIX Security 20), Boston, MA, USA.
34. Reddy, S., Lemieux, C., Padhye, R., and Sen, K. (2020, January 5–11). Quickly generating diverse valid test inputs with reinforcement learning. Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, Seoul, Republic of Korea.10.1145/3377811.3380399 ↗
35. Manès, V.J., Kim, S., and Cha, S.K. (2020, January 5–11). Ankou: Guiding grey-box fuzzing towards combinatorial difference. Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, Seoul, Republic of Korea.10.1145/3377811.3380421 ↗
36. Gan, S., Zhang, C., Chen, P., Zhao, B., Qin, X., Wu, D., and Chen, Z. (, January 12–14). GREYONE: Data flow sensitive fuzzing. Proceedings of the 29th USENIX Security Symposium (USENIX Security 20), Boston, MA, USA.
37. Pereira, J.C., Machado, N., and Sousa Pinto, J. (2020, January 22–23). Testing for race conditions in distributed systems via SMT solving. Proceedings of the International Conference on Tests and Proofs, Bergen, Norway.10.1007/978-3-030-50995-8_7 ↗
38. Wang, M., Srikant, S., Samak, M., and O’Reilly, U.M. (2023, January 17). RaceInjector: Injecting Races to Evaluate and Learn Dynamic Race Detection Algorithms. Proceedings of the 12th ACM SIGPLAN International Workshop on the State of the Art in Program Analysis, Orlando, FL, USA.10.1145/3589250.3596142 ↗
39. Hybrid static-dynamic analysis of data races caused by inconsistent locking discipline in device drivers
40. IRHunter: Universal Detection of Instruction Reordering Vulnerabilities for Enhanced Concurrency in Distributed and Parallel Systems10.1109/TPDS.2025.3556861 ↗