CSS: Container Resource Manager Using System Call Pattern for Scientific Workflow
Source abstract
Abstract
Multiple containers running scientific workflows in SMP-based high-performance computers generate some bottlenecks due to workload flexibility. To improve system resource utilization by minimizing these bottlenecks, vertical resource management is required to determine an appropriate resource usage policy according to the resource usage type of the container. However, the traditional methods have additional overhead for collecting monitoring metrics, and the structure of the resource manager is complex. In this paper, in order to compensate for these shortcomings, we propose CSS, a dynamic resource manager utilizing system call data collected for security purposes. The CSS utilizes the SBCC algorithm, which uses the number of futex system calls as a heuristic measure to determine the number of IO-intensive workload occurrences. In addition, the CTBRA algorithm is used to determine the range of resources to be allocated for each container and to perform actual resource allocation. We implemented a prototype of CSS and conducted experiments on NPB to analyze the performance of CSS with various types of large-scale tasks of a scientific workflow. As a result of the experiment, it showed a performance improvement of up to 7% compared with the environment where Linux cgroups were not applied. In addition, CANU performance analysis was performed to verify the effectiveness of applications used in the real world, and performance improvement of up to 4.5% was shown.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Distributed and Parallel Computing Systems · Software System Performance and Reliability · Cloud Computing and Resource Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Engineering, Korea University, Seoul 02841, Korea
Department of Computer Science, Dongduk Women’s University, Seoul 02748, Korea
Funding
National Research Foundation of Korea (NRF)
References (21)
Show source-provided references
1. Rao, V., Singh, V., Goutham, K.S., Kempaiah, B.U., Mampilli, R.J., Kalambur, S., and Sitaram, D. (2021, January 21). Scheduling Microservice Containers on Large Core Machines Through Placement and Coalescing. Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Virtual.10.1007/978-3-030-88224-2_5 ↗
2. Al-Dhuraibi, Y., Paraiso, F., Djarallah, N., and Merle, P. (2017, January 25–30). Autonomic vertical elasticity of docker containers with elasticdocker. Proceedings of the 10th International Conference on Cloud Computing (CLOUD), Honolulu, HI, USA.10.1109/CLOUD.2017.67 ↗
3. Runsewe, O., and Samaan, N. (2019, January 14–17). CRAM: A container resource allocation mechanism for big data streaming applications. Proceedings of the International Symposium on Cluster, Cloud and Grid Computing (CCGRID), Larnaca, Cyprus.10.1109/CCGRID.2019.00045 ↗
4. Hu, Y., de Laat, C.D., and Zhao, Z. (2019, January 14–17). Multi-objective container deployment on heterogeneous clusters. Proceedings of the International Symposium on Cluster, Cloud and Grid Computing (CCGRID), Larnaca, Cyprus.10.1109/CCGRID.2019.00076 ↗
5. Tan, B., Ma, H., and Mei, Y. (2020, January 11–14). A NSGA-II-based Approach for Multi-objective Micro-service Allocation in Container-based Clouds. Proceedings of the International Symposium on Cluster, Cloud and Internet Computing (CCGRID), Melbourne, Australia.10.1109/CCGrid49817.2020.00-65 ↗
6. Nikkhah, S.T., Geilen, M., Goswami, D., Koedam, M., Nelson, A., and Goossens, K. (2021, January 1–3). A Deployment Framework for Quality-Sensitive Applications in Resource-Constrained Dynamic Environments. Proceedings of the 24th Euromicro Conference on Digital System Design (DSD), Palermo, Italy.10.1109/DSD53832.2021.00042 ↗
7. Skarlatos, D., Chen, Q., Chen, J., Xu, T., and Torrellas, J. (2020, January 17–21). Draco: Architectural and operating system support for system call security. Proceedings of the International Symposium on Microarchitecture (MICRO), Athens, Greece.10.1109/MICRO50266.2020.00017 ↗
8. (2022, June 22). Cgroups Web Documentation. Available online: https://www.kernel.org/doc/documentation/cgroup-v1/cgroups.txt.
9. Kurtzer, G.M., Sochat, V., and Bauer, M.W. (2017). Singularity: Scientific containers for mobility of compute. PLoS ONE, 12.10.1371/journal.pone.0177459 ↗
10. (2022, June 22). Npb Website, Available online: https://www.nas.nasa.gov/software/npb.html.
11. Song, C., Gil, J., and Lim, J. (2022, January 19–21). A Performance Analysis on HPC Task Using cgroups in Singularity Container Runtime Environment. Proceedings of the KIPS Annual Spring Conference, Seoul, Korea.
12. Feeley, M.J., Chase, J.S., and Lazowska, E.D. (1993). User-Level Threads and Interprocess Communication, University of Washington, Department of Computer Science and Engineering. Technical Report 93-02-03.
13. The vision of autonomic computing10.1109/MC.2003.1160055 ↗
14. (2022, June 22). Bpftrace Github Repository. Available online: https://github.com/iovisor/bpftrace.
15. (2022, June 22). Containerd Website. Available online: https://containerd.io.
16. (2022, June 22). PacBio DevNet Website. Available online: http://pacbiodevnet.com.
17. Ghavamnia, S., Palit, T., and Benameur, A. (2020, January 14–15). Confine: Automated system call policy generation for container attack surface reduction. Proceedings of the 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID), San Sebastian, Spain.
18. Wang, X., Shen, Q., Luo, W., and Wu, P. (2020, January 18–24). RSDS: Getting System Call Whitelist for Container Through Dynamic and Static Analysis. Proceedings of the 13th IEEE International Conference on Cloud Computing (CLOUD), Virtual Event.10.1109/CLOUD49709.2020.00089 ↗
19. Kim, S., Kim, B., and Lee, D. (2021, January 5–10). Prof-gen: Practical Study on System Call Whitelist Generation for Container Attack Surface Reduction. Proceedings of the 14th IEEE International Conference on Cloud Computing, (CLOUD), Chicago, IL, USA.10.1109/CLOUD53861.2021.00041 ↗
20. Russo, G.R., Cardellini, V., Casale, G., and Presti, F.L. (2021, January 10–13). MEAD: Model-Based Vertical Auto-Scaling for Data Stream Processing. Proceedings of the International Symposium on Cluster, Cloud and Internet Computing (CCGRID), Melbourne, Australia.10.1109/CCGrid51090.2021.00041 ↗
21. Hobson, T., Yildiz, O., Nicolae, B., Huang, J., and Peterka, T. (2021, January 10–13). Shared-Memory Communication for Containerized Workflows. Proceedings of the International Symposium on Cluster, Cloud and Internet Computing (CCGRID), Melbourne, Australia.10.1109/CCGrid51090.2021.00022 ↗