A Multi-Task Dynamic Weight Optimization Framework Based on Deep Reinforcement Learning
Source abstract
Abstract
In recent years, multi-task learning (MTL) has been shown to have considerable potential for enhancing model performance through the inter-task sharing of knowledge. Nevertheless, the effective balancing of optimization across diverse tasks remains a significant challenge. This paper presents a novel Deep Reinforcement Learning (DRL)-based framework that employs an Actor-Critic architecture to dynamically adjust task weights. In contrast to traditional methodologies that depend on heuristic rules or predefined assumptions about inter-task relationships, our approach autonomously learns optimal weighting strategies through interactions between an intelligent agent and the multi-task environment. The experimental results obtained across multiple datasets demonstrate that our method not only enhances overall performance and achieves a more balanced distribution of tasks but also maintains high training efficiency. Furthermore, the framework’s robust generalization and adaptability make it well-suited for addressing complex and dynamic learning scenarios.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Human Pose and Action Recognition · Autonomous Vehicle Technology and Safety
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science, China University of Geosciences, Wuhan 430074, China
Informatization Office, China University of Geosciences, Wuhan 430074, China
Funding
University-Industry Collaborative Education Program of the Ministry of Education of China
References (40)
Show source-provided references
1. Multitask learning10.1023/A:1007379606734 ↗
2. Miettinen, K. (1999). Nonlinear Multiobjective Optimization, Springer Science & Business Media.10.1007/978-1-4615-5563-6 ↗
3. A proposal on machine learning via dynamical systems
4. Stable attractors for neural networks classification via ordinary differential equations (SA-nODE)10.1088/2632-2153/ad7f26 ↗
5. Geshkovski, B., Letrouit, C., Polyanskiy, Y., and Rigollet, P. (2023). A mathematical perspective on transformers. arXiv.
6. Steepest descent methods for multicriteria optimization10.1007/s001860000043 ↗
7. Coello, C.A.C., Aguirre, A.H., and Zitzler, E. (2005). Evolutionary Multi-Criterion Optimization, Springer.
8. Misra, I., Shrivastava, A., Gupta, A., and Hebert, M. (2016, January 27–30). Cross-stitch networks for multi-task learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.433 ↗
9. Rusu, A.A., Rabinowitz, N.C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., and Hadsell, R. (2016). Progressive neural networks. arXiv.
10. Ruder, S., Bingel, J., Augenstein, I., and Søgaard, A. (February, January 27). Latent multi-task architecture learning. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
11. Gao, Y., Ma, J., Zhao, M., Liu, W., and Yuille, A.L. (2019, January 15–20). NDDR-CNN: Layerwise feature fusing in multi-task CNNs by neural discriminative dimensionality reduction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00332 ↗
12. Hai, Z., Zhao, P., Cheng, P., Yang, P., Li, X.-L., and Li, G. (2016, January 1–5). Deceptive review spam detection via exploiting task relatedness and unlabeled data. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, TX, USA.10.18653/v1/D16-1187 ↗
13. Guo, H., Pasunuru, R., and Bansal, M. (2018). Soft layer-specific multi-task summarization with entailment and question generation. arXiv.10.18653/v1/P18-1064 ↗
14. Lan, M., Wang, J., Wu, Y., Niu, Z.-Y., and Wang, H. (2017, January 7–11). Multi-task attention-based neural networks for implicit discourse relationship representation and identification. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.10.18653/v1/D17-1134 ↗
15. Dankers, V., Rei, M., Lewis, M., and Shutova, E. (2019, January 3–7). Modelling the interplay of metaphor and emotion through multitask learning. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Hong Kong, China.10.18653/v1/D19-1227 ↗
16. Huang, J., Feris, R.S., Chen, Q., and Yan, S. (2015, January 7–13). Cross-domain image retrieval with a dual attribute-aware ranking network. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.127 ↗
17. Hyperface: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition10.1109/TPAMI.2017.2781233 ↗
18. Jou, B., and Chang, S.-F. (2016, January 15–19). Deep cross residual learning for multitask visual recognition. Proceedings of the 24th ACM International Conference on Multimedia, Amsterdam, The Netherlands.10.1145/2964284.2964309 ↗
19. Bilen, H., and Vedaldi, A. (2016). Integrated perception with recurrent multi-task neural networks. Advances in Neural Information Processing Systems, Proceedings of the Annual Conference on Neural Information Processing Systems 2016, Barcelona, Spain, 5–10 December 2016, Curran Associates Inc.
20. Kokkinos, I. (2017, January 21–26). Ubernet: Training a universal convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.579 ↗
21. Dvornik, N., Shmelkov, K., Mairal, J., and Schmid, C. (2017, January 22–29). Blitznet: A real-time deep network for scene understanding. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.447 ↗
22. Guo, P., Lee, C.-Y., and Ulbricht, D. (2020, January 12–18). Learning to branch for multi-task learning. Proceedings of the International Conference on Machine Learning, Vienna, Austria.
23. Zhang, Y., and Yeung, D.-Y. (2012). A convex formulation for learning task relationships in multi-task learning. arXiv.
24. Kendall, A., Gal, Y., and Cipolla, R. (2018, January 18–22). Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.
25. Guo, M., Haque, A., Huang, D.-A., Yeung, S., and Li, F.-F. (2018, January 8–14). Dynamic task prioritization for multitask learning. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01270-0_17 ↗
26. Pareto-path multitask multiple kernel learning10.1109/TNNLS.2014.2309939 ↗
27. Sener, O., and Koltun, V. (2018). Multi-task learning as multi-objective optimization. Advances in Neural Information Processing Systems, Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, QC, Canada, 3–8 December 2018, Curran Associates Inc.
28. Chen, Z., Badrinarayanan, V., Lee, C.-Y., and Rabinovich, A. (2018, January 10–15). GradNorm: Gradient normalization for adaptive loss balancing in deep multitask networks. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
29. Hotegni, S.S., Berkemeier, M., and Peitz, S. (July, January 30). Multi-objective optimization for sparse deep multi-task learning. Proceedings of the 2024 International Joint Conference on Neural Networks (IJCNN), Yokohama, Japan.10.1109/IJCNN60899.2024.10650994 ↗
30. Asadi, S., Gharibzadeh, S., Zangeneh, S., Reihanifar, M., Rahimi, M., and Abdullah, L. (2024). Comparative analysis of gradient-based optimization techniques using multidimensional surface 3D visualizations and initial point sensitivity. arXiv.
31. Vithayathil Varghese, N., and Mahmoud, Q.H. (2020). A survey of multi-task deep reinforcement learning. Electronics, 9.10.3390/electronics9091363 ↗
32. Multi-task reinforcement learning in humans10.1038/s41562-020-01035-y ↗
33. Borsa, D., Graepel, T., and Shawe-Taylor, J. (2016). Learning shared representations in multi-task reinforcement learning. arXiv.
34. Mastering the game of Go with deep neural networks and tree search10.1038/nature16961 ↗
35. Fernando, C., Banarse, D., Blundell, C., Zwols, Y., Ha, D., Rusu, A.A., Pritzel, A., and Wierstra, D. (2017). PathNet: Evolution channels gradient descent in super neural networks. arXiv.
36. Rosenbaum, C., Klinger, T., and Riemer, M. (2017). Routing networks: Adaptive selection of non-linear functions for multi-task learning. arXiv.
37. PACO: Parameter-compositional multi-task reinforcement learning
38. Vuong, T.-L., Nguyen, D.-V., Nguyen, T.-L., Bui, C.-M., Kieu, H.-D., Ta, V.-C., Tran, Q.-L., and Le, T.-H. (2019, January 10–16). Sharing experience in multitask reinforcement learning. Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China.10.24963/ijcai.2019/505 ↗
39. Sabour, S., Frosst, N., and Hinton, G.E. (2017). Dynamic routing between capsules. Advances in Neural Information Processing Systems, Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4–7 December 2017, Curran Associates Inc.
40. Liu, S., Johns, E., and Davison, A.J. (2019, January 15–20). End-to-end multi-task learning with attention. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00197 ↗