journal article·2021-07-02Open access

Improvements to Supercomputing Service Availability Based on Data Analysis

Jae-Kook Lee, Min-Woo Kwon, Do-Sik An, Junweon Yoon, Taeyoung Hong, Joon Woo, Sung-Jun Kim, Guohua Li

Applied Sciences· Volume 11· Issue 13

Abstract

As the demand for high-performance computing (HPC) resources has increased in the field of computational science, an inevitable consideration is service availability in large cluster systems such as supercomputers. In particular, the factor that most affects availability in supercomputing services is the job scheduler utilized for allocating resources. Consequent to submitting user data through the job scheduler for data analysis, 25.6% of jobs failed because of program errors, scheduler errors, or I/O errors. Based on this analysis, we propose a K-hook method for scheduling to increase the success rate of job submissions and improve the availability of supercomputing services. By applying this method, the job-submission success rate was improved by 15% without negatively affecting users’ waiting time. We also achieved a mean time between interrupts (MTBI) of 24.3 days and maintained average system availability at 97%. As this research was verified on the Nurion supercomputer in a real service environment, the value of the research is expected to be found in significant service improvements.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Cloud Computing and Resource Management · Distributed and Parallel Computing Systems · Advanced Data Storage Technologies

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

National Supercomputing Center, Korea Institute of Science and Technology Information, 245 Daehak-ro, Yuseong-gu, Daejeon 34141, Korea

References (15)

Show source-provided references
1. Di, S., Guo, H., Pershey, E., Snir, M., and Cappello, F. (2019, January 24–27). Characterizing and Understanding HPC Job Failures over the 2K-day Life of IBM BlueGene/Q System. Proceedings of the 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), Portland, OR, USA.10.1109/DSN.2019.00055 ↗
2. Exploring properties and correlations of fatal events in a large-scale hpc system10.1109/TPDS.2018.2864184 ↗
3. Rojas, E., Meneses, E., Jones, T., and Maxwell, D. (2019, January 15–18). Analyzing a five-year failure record of a leadership-class supercomputer. Proceedings of the 31st International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), Campo Grande, Brazil.10.1109/SBAC-PAD.2019.00040 ↗
4. Wang, F., Oral, S., Sen, S., and Imam, N. (2019, January 23–26). Learning from Five-year Resource-Utilization Data of Titan System. Proceedings of the 2019 IEEE International Conference on Cluster Computing (CLUSTER), Albuquerque, NM, USA.10.1109/CLUSTER.2019.8891001 ↗
5. The K computer operations: Experiences and statistics10.1016/j.procs.2014.05.052 ↗
6. Wang, C., Mueller, F., Engelmann, C., and Scott, S.L. (2008, January 15–21). Proactive process-level live migration in HPC environments. Proceedings of the SC’08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing, Austin, TX, USA.
7. Hsu, C.H., and Feng, W.C. (2005, January 12–18). A power-aware run-time system for high-performance computing. Proceedings of the SC’05: Proceedings of the 2005 ACM/IEEE Conference on Supercomputing, Seattle, WA, USA.
8. A survey of high-performance computing scaling challenges10.1177/1094342015597083 ↗
9. Yoon, J., Hong, T., Park, C., Noh, S.Y., and Yu, H. (2020). Log Analysis-Based Resource and Execution Time Improvement in HPC: A Case Study. Appl. Sci., 10.10.3390/app10072634 ↗
10. Wagner, M., López, V., Morillo, J., Cavazzoni, C., Affinito, F., Giménez, J., and Labarta, J. (2017, January 14–17). Performance analysis and optimization of the fftxlib on the intel knights landing architecture. Proceedings of the 46th International Conference on Parallel Processing Workshops (ICPPW), Bristol, UK.10.1109/ICPPW.2017.44 ↗
11. Kang, J.H., Kwon, O.K., Ryu, H., Jeong, J., and Lim, K. (2018, January 16–20). Performance evaluation of scientific applications on Intel Xeon Phi Knights Landing clusters. Proceedings of the 2018 International Conference on High Performance Computing & Simulation (HPCS), Orleans, France.10.1109/HPCS.2018.00063 ↗
12. Hammond, S., Vaughan, C., and Hughes, C. (2018, January 16–20). Evaluating the Intel Skylake Xeon processor for HPC workloads. Proceedings of the 2018 International Conference on High Performance Computing & Simulation (HPCS), Orleans, France.10.1109/HPCS.2018.00064 ↗
13. Birrittella, M.S., Debbage, M., Huggahalli, R., Kunz, J., Lovett, T., Rimmer, T., Underwood, K.D., and Zak, R.C. (2015, January 26–28). Intel® Omni-path architecture: Enabling scalable, high performance fabrics. Proceedings of the IEEE 23rd Annual Symposium on High-Performance Interconnects, Santa Clara, CA, USA.10.1109/HOTI.2015.22 ↗
14. Salunkhe, R., Kadam, A.D., Jayakumar, N., and Joshi, S. (2016, January 3–5). Luster a scalable architecture file system: A research implementation on active storage array framework with Luster file system. Proceedings of the International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT), Chennai, India.10.1109/ICEEOT.2016.7754852 ↗
15. Kim, J., and Kim, D. (2011, January 21–23). KREONET-GENI future internet testbed. Proceedings of the 7th International Conference on Networked Computing and Advanced Information Management, Gyeongju, Korea.