Entropy-Guided Hierarchical Scheduling for Elastic Distributed Deep Learning
Source abstract
Abstract
Shared GPU clusters often execute multiple distributed training jobs concurrently under fluctuating contention. We reinterpret this setting as a two-scale control problem, where the micro scale captures intra-job learning dynamics and the macro scale captures inter-job resource arbitration. We propose an entropy-guided hierarchical framework that links these two scales through a unified uncertainty signal computed from training logits. Unlike existing uncertainty-aware methods that typically use uncertainty for only a single level of decision making, our approach uses the same entropy-based signal to jointly support both intra-job adaptation and inter-job scheduling within a hierarchical control loop. At the micro level, each worker estimates predictive uncertainty via normalized entropy and converts it into stable weights that drive epoch-level controls for uncertainty-aware data sharding, fixed-budget batch-size reallocation, and learning-rate modulation, while remaining compatible with standard synchronous data-parallel training. At the macro level, the same signal is aggregated into a job utility score that guides admission, ordering, and GPU quota assignment under contention. In large-scale workload-driven simulation, our method reduces average job completion time (JCT) by 23.7% and shortens cluster makespan by 15.7% relative to a strong learning-unaware baseline, demonstrating that uncertainty-aligned scheduling can improve cluster-level efficiency while preserving training correctness. We further validate scalability using a calibrated simulator up to 1024 nodes.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Cloud Computing and Resource Management · Stochastic Gradient Optimization Techniques · Distributed and Parallel Computing Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Artificial Intelligence, Kyung Hee University, Yongin 17104, Republic of Korea
Department of Computer Science and Engineering, Kyung Hee University, Yongin 17104, Republic of Korea
Artificial Intelligence Research Center, Kyung Hee University, Yongin 17104, Republic of Korea
Funding
MSIT (Ministry of Science and ICT), Korea · IITP · Global—Learning & Academic Research Institution · National Research Foundation of Korea
References (53)
Show source-provided references
1. Deep learning workload scheduling in gpu datacenters: A survey
2. Peng, Y., Bao, Y., Chen, Y., Wu, C., and Guo, C. (2018, January 23–26). Optimus: An efficient dynamic resource scheduler for deep learning clusters. Proceedings of the Thirteenth EuroSys Conference, Porto, Portugal.10.1145/3190508.3190517 ↗
3. Mahajan, K., Balasubramanian, A., Singhvi, A., Venkataraman, S., Akella, A., Phanishayee, A., and Chawla, S. (2020, January 25–27). Themis: Fair and efficient GPU cluster scheduling. Proceedings of the 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20), Santa Clara, CA, USA.
4. Xiao, W., Bhardwaj, R., Ramjee, R., Sivathanu, M., Kwatra, N., Han, Z., Patel, P., Peng, X., Zhao, H., and Zhang, Q. (2018, January 8–10). Gandiva: Introspective cluster scheduling for deep learning. Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), Carlsbad, CA, USA.
5. Gu, J., Chowdhury, M., Shin, K.G., Zhu, Y., Jeon, M., Qian, J., Liu, H., and Guo, C. (2019, January 26–28). Tiresias: A GPU cluster manager for distributed deep learning. Proceedings of the 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19), Boston, MA, USA.
6. Zheng, P., Pan, R., Khan, T., Venkataraman, S., and Akella, A. (2023, January 17–19). Shockwave: Fair and efficient cluster scheduling for dynamic adaptation in machine learning. Proceedings of the 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), Boston, MA, USA.
7. Astraea: A fair deep learning scheduler for multi-tenant gpu clusters
8. Kaur, R., Asad, A., Al Abdul Wahid, S., and Mohammadi, F. (2025). A Survey of Advancements in Scheduling Techniques for Efficient Deep Learning Computations on GPUs. Electronics, 14.10.20944/preprints202412.0276.v2 ↗
9. Hu, Q., Ye, Z., Wang, Z., Wang, G., Zhang, M., Chen, Q., Sun, P., Lin, D., Wang, X., and Luo, Y. (2024, January 16–18). Characterization of large language model development in the datacenter. Proceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), Santa Clara, CA, USA.
10. Qiao, A., Choe, S.K., Subramanya, S.J., Neiswanger, W., Ho, Q., Zhang, H., Ganger, G.R., and Xing, E.P. (2021, January 14–16). Pollux: Co-adaptive cluster scheduling for goodput-optimized deep learning. Proceedings of the 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21), Virtual.
11. Li, J., Xu, H., Zhu, Y., Liu, Z., Guo, C., and Wang, C. (2023, January 8–11). Lyra: Elastic scheduling for deep learning clusters. Proceedings of the Eighteenth European Conference on Computer Systems, Rome, Italy.10.1145/3552326.3587445 ↗
12. Sharma, A., Bhasi, V.M., Singh, S., Kesidis, G., Kandemir, M.T., and Das, C.R. (2024). Gpu cluster scheduling for network-sensitive deep learning. arXiv.
13. Strati, F., Ma, X., and Klimovic, A. (2024, January 23–25). Orion: Interference-aware, fine-grained gpu sharing for ml applications. Proceedings of the Nineteenth European Conference on Computer Systems, Athens, Greece.10.1145/3627703.3629578 ↗
14. Weng, Q., Yang, L., Yu, Y., Wang, W., Tang, X., Yang, G., and Zhang, L. (2023, January 10–12). Beware of fragmentation: Scheduling GPU-Sharing workloads with fragmentation gradient descent. Proceedings of the 2023 USENIX Annual Technical Conference (USENIX ATC 23), Santa Clara, CA, USA.
15. Yang, Z.Y., Xia, W.K., Chu, H.Q., Su, W.H., Wang, R.F., and Wang, H. (2025). A comprehensive review of deep learning applications in cotton industry: From field monitoring to smart processing. Plants, 14.10.3390/plants14101481 ↗
16. Wu, A.Q., Li, K.L., Song, Z.Y., Lou, X., Hu, P., Yang, W., and Wang, R.F. (2025). Deep learning for sustainable aquaculture: Opportunities and challenges. Sustainability, 17.10.3390/su17115084 ↗
17. Chen, J., Pan, X., Monga, R., Bengio, S., and Jozefowicz, R. (2016). Revisiting distributed synchronous SGD. arXiv.
18. Sergeev, A., and Del Balso, M. (2018). Horovod: Fast and easy distributed deep learning in TensorFlow. arXiv.
19. Zhang, H., Zheng, Z., Xu, S., Dai, W., Ho, Q., Liang, X., Hu, Z., Wei, J., Xie, P., and Xing, E.P. (2017, January 12–14). Poseidon: An efficient communication architecture for distributed deep learning on GPU clusters. Proceedings of the 2017 USENIX Annual Technical Conference (USENIX ATC 17), Santa Clara, CA, USA.
20. Jiang, Y., Zhu, Y., Lan, C., Yi, B., Cui, Y., and Guo, C. (2020, January 4–6). A unified architecture for accelerating distributed DNN training in heterogeneous GPU/CPU clusters. Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), Virtual.
21. A Survey on Parameter Server Architecture: Approaches for Optimizing Distributed Centralized Learning10.1109/ACCESS.2025.3535085 ↗
22. Liang, F., Zhang, Z., Lu, H., Leung, V., Guo, Y., and Hu, X. (2024). Communication-efficient large-scale distributed deep learning: A comprehensive survey. arXiv.
23. Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., and He, K. (2017). Accurate, large minibatch sgd: Training imagenet in 1 hour. arXiv.
24. McCandlish, S., Kaplan, J., Amodei, D., and OAID Team (2018). An empirical model of large-batch training. arXiv.
25. Smith, S.L., Kindermans, P.J., Ying, C., and Le, Q.V. (2017). Don’t decay the learning rate, increase the batch size. arXiv.
26. You, Y., Li, J., Reddi, S., Hseu, J., Kumar, S., Bhojanapalli, S., Song, X., Demmel, J., Keutzer, K., and Hsieh, C.-J. (2019). Large batch optimization for deep learning: Training bert in 76 minutes. arXiv.
27. Settles, B. (2009). Active Learning Literature Survey, University of Wisconsin-Madison.
28. Gal, Y., and Ghahramani, Z. (2016, January 19–24). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA.
29. Unlocking the black box: An in-depth review on interpretability, explainability, and reliability in deep learning10.1007/s00521-024-10437-2 ↗
30. Fort, S., Hu, H., and Lakshminarayanan, B. (2019). Deep ensembles: A loss landscape perspective. arXiv.
31. Ensemble deep learning: A review10.1016/j.engappai.2022.105151 ↗
32. A review of ensemble learning and data augmentation models for class imbalanced problems: Combination, implementation and evaluation10.1016/j.eswa.2023.122778 ↗
33. Data generation for machine learning interatomic potentials and beyond10.1021/acs.chemrev.4c00572 ↗
34. A survey on scheduling techniques in computing and network convergence10.1109/COMST.2023.3329027 ↗
35. Liang, F., Zhang, Z., Lu, H., Li, C., Leung, V., Guo, Y., and Hu, X. (2024). Resource allocation and workload scheduling for large-scale distributed deep learning: A survey. arXiv.
36. Wu, B., Zhong, Y., Zhang, Z., Liu, S., Liu, F., Sun, Y., Huang, G., Liu, X., and Jin, X. (2023). Fast distributed inference serving for large language models. arXiv.
37. Zhang, Y.K., Zhan, D.C., and Ye, H.J. (2025). Capability Instruction Tuning. Proceedings of the AAAI Conference on Artificial Intelligence, AAAI Press.10.1609/aaai.v39i24.34790 ↗
38. Choudhury, A., Wang, Y., Pelkonen, T., Srinivasan, K., Jain, A., Lin, S., David, D., Soleimanifard, S., Chen, M., and Yadav, A. (2024, January 10–12). MAST: Global scheduling of ML training across Geo-Distributed datacenters at hyperscale. Proceedings of the 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), Santa Clara, CA, USA.
39. On efficient training of large-scale deep learning models10.1145/3700439 ↗
40. Tuning large neural networks via zero-shot hyperparameter transfer
41. Balles, L., Romero, J., and Hennig, P. (2016). Coupling Adaptive Batch Sizes with Learning Rates. arXiv.
42. He, F., Liu, T., and Tao, D. (2019). Control batch size and learning rate to generalize well: Theoretical and empirical evidence. Proceedings of the Advances in Neural Information Processing Systems, Curran Associates, Inc.
43. A survey on failure analysis and fault injection in AI systems
44. Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J., Lakshminarayanan, B., and Snoek, J. (2019). Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift. Proceedings of the Advances in Neural Information Processing Systems, Curran Associates, Inc.
45. Mind the gap: Challenges of deep learning approaches to theory of mind10.1007/s10462-023-10401-x ↗
46. Deep Bayesian Active Learning with Image Data
47. Fakour, F., Mosleh, A., and Ramezani, R. (2024). A structured review of literature on uncertainty in machine learning & deep learning. arXiv.
48. A survey on uncertainty quantification methods for deep learning
49. Lakshminarayanan, B., Pritzel, A., and Blundell, C. (2017). Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Curran Associates, Inc.
50. A Survey of Uncertainty in Deep Neural Networks10.1007/s10462-023-10562-9 ↗
51. PHH-FL: Perceptual Hashing Hypernetwork Personalized Federated Learning for Heterogeneous Medical Image Analysis Tasks10.1109/JIOT.2025.3641888 ↗
52. Knowledge distillation and teacher-student learning in medical imaging: Comprehensive overview, pivotal role, and future directions
53. Vision-Language Models in medical image analysis: From simple fusion to general large models10.1016/j.inffus.2025.102995 ↗