journal article·2025-01-06Open access

Design and Implementation of a Reconfigurable Test Environment for Network Measurement Tools Based on a Control and Management Framework

Fatih Abut, Mehmet Kızıldağ

Applied Sciences· Volume 15· Issue 1

Abstract

To be applicable to real-world networks, a realistic evaluation of network measurement tools requires the conditions of the test network to be as similar as possible to the networks in which they are later deployed. Thus, having an existing real-world network which can be utilized for testing, while also allowing the reconfiguration of certain characteristics and features, is of significant interest to researchers. Providing the researchers with corresponding management tools with which they can introduce artificial conditions and topology changes into the network would help in realizing more realistic testbed setups and scenarios. The purpose of this study is to develop a configurable test environment based on the Control and Management Framework (OMF) to support researchers in a realistic empirical evaluation of network measurement tools. The proposed OMF-enabled test environment has the merit of automatizing the deployments and executions of any measurement experiments, collecting their final estimates, and then transporting them back to a single experiment controller without requiring any physical access to the corresponding experimental nodes used. It is demonstrated how the test environment can be used for the evaluation of network measurement experiments based on four representative case studies. These include validating the accuracy of end-to-end round-trip time and available bandwidth estimation tools, measuring the TCP and UDP throughput, and analyzing the impact of packet loss and jitter on VoIP performance, all conducted over dynamically adjusted testbed configurations. These case studies highlight the versatility and effectiveness of the OMF-enabled test environment, positioning it as a valuable tool for advancing the realistic evaluation of network measurement tools in controlled yet adaptable settings.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Software-Defined Networks and 5G · Software System Performance and Reliability · Interconnection Networks and Systems

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Computer Engineering, Faculty of Engineering, Çukurova University, 01330 Adana, Turkey

SABIS Bilgi Teknolojileri ve San. Tic. Ltd. Şti, 01330 Adana, Turkey

References (31)

Show source-provided references
1. Sun, M., He, B., Li, R., Li, J., and Zhang, X. (2023). A Survey: Network Feature Measurement Based on Machine Learning. Appl. Sci., 13.10.3390/app13042551 ↗
2. A Comparative Analysis of Ookla Speedtest and Measurement Labs Network Diagnostic Test (NDT7)10.1145/3579448 ↗
3. Labit, Y., Owezarski, P., and Larrieu, N. (2005, January 15). Evaluation of Active Measurement Tools for Bandwidth Estimation in Real Environment. Proceedings of the Workshop on End-to-End Monitoring Techniques and Services, Nice, France.
4. NetSim—The Framework for Complex Network Generator10.1016/j.procs.2018.07.289 ↗
5. Kashihara, S., Tamai, M., Hasegawa, A., and Yokoyama, H. (2023). Multilayer Measurement Methodology with Open-Source Tools for Evaluating IEEE 802.11ad Communication Quality. Appl. Sci., 13.10.3390/app13095378 ↗
6. Mazhar, T., Malik, M.A., Mohsan, S.A.H., Li, Y., Haq, I., Ghorashi, S., Karim, F.K., and Mostafa, S.M. (2023). Quality of Service (QoS) Performance Analysis in a Traffic Engineering Model for Next-Generation Wireless Sensor Networks. Symmetry, 15.10.3390/sym15020513 ↗
7. Traffic Engineering and Quality of Service in Hybrid Software Defined Networks10.23919/JCC.fa.2022-0860.202402 ↗
8. Through the Diversity of Bandwidth-Related Metrics, Estimation Techniques and Tools: An Overview
9. A Survey of Techniques for Internet Topology Discovery10.1109/COMST.2014.2376520 ↗
10. Survey on the End-to-End Internet Delay Measurements10.1007/978-3-540-25969-5_14 ↗
11. OMF: A Control and Management Framework for Networking Testbeds10.1145/1713254.1713267 ↗
12. Chen, Z., Zhao, Z., Li, Z., Shao, J., Liu, S., and Xu, Y. (November, January 31). SDT: A Low-Cost and Topology-Reconfigurable Testbed for Network Research. Proceedings of the IEEE Computer Society, Santa Fe, NM, USA.
13. A Modular, End-to-End Next-Generation Network Testbed: Toward a Fully Automated Network Management Platform10.1109/TNSM.2024.3416031 ↗
14. Callebaut, G., Mulders, J.V., Ottoy, G., Delabie, D., Cox, B., Stevens, N., and der Perre, L.V. (2022, January 7–10). Techtile—Open 6G R&D Testbed for Communication, Positioning, Sensing, WPT and Federated Learning. Proceedings of the 2022 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), Grenoble, France.10.1109/EuCNC/6GSummit54941.2022.9815696 ↗
15. Morano, G., Hrovat, A., Vučnik, M., Puhan, J., Gardašević, G., Vasiljević, D., and Javornik, T. (2021). Experiment Control and Monitoring System for LOG-a-TEC Testbed. Sensors, 21.10.3390/s21196422 ↗
16. Muelas, D., Ramos, J., and López de Vergara, J.E. (2018). Software-Driven Definition of Virtual Testbeds to Validate Emergent Network Technologies. Information, 9.10.3390/info9020045 ↗
17. Adjih, C., Baccelli, E., Fleury, E., Harter, G., Mitton, N., Noël, T., Pissard-Gibollet, R., Saint-Marcel, F., Schreiner, G., and Vandaele, J. (2015, January 14–16). FIT IoT-LAB: A Large Scale Open Experimental IoT Testbed. Proceedings of the 2015 IEEE 2nd World Forum on Internet of Things (WF-IoT), Milan, Italy.10.1109/WF-IoT.2015.7389098 ↗
18. Design and Implementation of a Versatile OpenHAB IoT Testbed with a Variety of Wireless Interfaces and Sensors10.3390/telecom4030026 ↗
19. Candell, R., Stouffer, K., and Anand, D. (2014, January 6–9). A Cybersecurity Testbed for Industrial Control Systems. Proceedings of the ISA Symposium on Process Control and Safety, Houston, TX, USA.
20. Pospisil, O., Fujdiak, R., Mikhaylov, K., Ruotsalainen, H., and Misurec, J. (2021). Testbed for LoRaWAN Security: Design and Validation through Man-in-the-Middle Attacks Study. Appl. Sci., 11.10.3390/app11167642 ↗
21. Sharma, S., Urumkar, S., Fontanesi, G., Ramamurthy, B., and Nag, A. (2022). Future Wireless Networking Experiments Escaping Simulations. Future Internet, 14.10.3390/fi14040120 ↗
22. Buzura, S., Peculea, A., Iancu, B., Cebuc, E., Dadarlat, V., and Kovacs, R. (2023). A Hybrid Software and Hardware SDN Simulation Testbed. Sensors, 23.10.3390/s23010490 ↗
23. Sørensen, C.W., Hernández Marcano, N.J., Cabrera Guerrero, J.A., Wunderlich, S., Lucani, D.E., and Fitzek, F.H.P. (2016). Easy as Pi: A Network Coding Raspberry Pi Testbed. Electronics, 5.10.3390/electronics5040067 ↗
24. Chamran, M.K., Yau, K.-L.A., Noor, R.M.D., and Wong, R. (2020). A Distributed Testbed for 5G Scenarios: An Experimental Study. Sensors, 20.10.3390/s20010018 ↗
25. Pathload: A Measurement Tool for End-to-End Available Bandwidth10.1109/TNET.2003.815304 ↗
26. Leischner, M. (2024, October 15). A Tool for Measuring End-to-End Round-Trip Time over TCP Connections. Available online: http://www.leischner.inf.fh-rhein-sieg.de/rtt-tool.
27. Passive Estimation of TCP Round-Trip Times10.1145/571697.571725 ↗
28. Strauss, J., Katabi, D., and Kaashoek, F. (2003, January 27). A Measurement Study of Available Bandwidth Estimation Tools. Proceedings of the 3rd ACM SIGCOMM Conference on Internet Measurement, New York, NY, USA.10.1145/948209.948211 ↗
29. Ribeiro, V., Riedi, R., Navrátil, J., and Cottrell, L. (2003). pathChirp: Efficient Available Bandwidth Estimation for Network Paths. Passiv. Act. Meas. Workshop, 1–11.
30. Li, M., Claypool, M., and Kinicki, R. (2008, January 14–17). WBest: A Bandwidth Estimation Tool for IEEE 802.11 Wireless Networks. Proceedings of the 33rd IEEE Conference on Local Computer Networks (LCN), Montreal, QC, Canada.
31. An Experimental Evaluation of Tools for Estimating Bandwidth-Related Metrics