Meta-Data-Guided Robust Deep Neural Network Classification with Noisy Label
Source abstract
Abstract
Deep neural network (DNN)-based classifiers have witnessed great applications in various fields. Unfortunately, the labels of real-world training data are commonly noisy, i.e., the labels of a large percentage of training samples are wrong, which negatively affects the performance of a trained DNN classifier during inference. Therefore, it is challenging to practically formulate a robust DNN classifier using noisy labels in training. To address the above issue, our work designs an effective architecture for training the robust DNN classifier with noisy labels, named a cross dual-branch network guided by meta-data on a single side (CoNet-MS), in which a small amount of clean data, i.e., meta-data, are used to guide the training of the DNN classifier. Specifically, the contributions of our work are threefold. First, based on the principle of small loss, each branch using the base classifier as a neural network module infers partial samples with pseudo-clean labels, which are then used for training another branch through a cross structure that can alleviate the cumulative impact of mis-inference. Second, a meta-guided module is designed and inserted into the single branch, e.g., the upper branch, which dynamically adjusts the ratio between the observed label and the pseudo-label output by the classifier in the loss function for each training sample. The asymmetric dual-branch design makes two classifiers diverge, which facilitates them to filter different types of noisy labels and avoid confirmation bias in self-training. Finally, thorough experiments demonstrate that the trained classifier with our proposal is more robust: the accuracy of the classifier trained with our proposed CoNet-MS on multiple datasets under various ratios of noisy labels and noise types outperforms other classifiers of learning with noisy labels (LNLs), including the state-of-the-art meta-data-based LNL classifier.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning and Data Classification
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Portland Institute, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
Faculty of Computer & Information Sciences, Hosei University, Tokyo 184-8584, Japan
Faculty of Human Sciences, Waseda University, Tokorozawa-shi, Saitama 359-1192, Japan
References (34)
Show source-provided references
1. Learning from noisy labels with deep neural networks: A survey10.1109/TNNLS.2022.3152527 ↗
2. Image classification with deep learning in the presence of noisy labels: A survey10.1016/j.knosys.2021.106771 ↗
3. Early-learning regularization prevents memorization of noisy labels
4. Yao, Y., Sun, Z., Zhang, C., Shen, F., Wu, Q., Zhang, J., and Tang, Z. (2021, January 20–25). Jo-src: A contrastive approach for combating noisy labels. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00515 ↗
5. Dgani, Y., Greenspan, H., and Goldberger, J. (2018, January 4–7). Training a neural network based on unreliable human annotation of medical images. Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA.10.1109/ISBI.2018.8363518 ↗
6. Cheng, J., Liu, T., Ramamohanarao, K., and Tao, D. (2020, January 13–18). Learning with bounded instance and label-dependent label noise. Proceedings of the International Conference on Machine Learning, Vienna, Austria.
7. Jiang, L., Zhou, Z., Leung, T., Li, L.-J., and Fei-Fei, L. (2018, January 10–15). Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
8. Arpit, D., Jastrzkebski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M.S., Maharaj, T., Fischer, A., Courville, A., and Bengio, Y. (2017, January 6–11). A closer look at memorization in deep networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia.
9. Co-teaching: Robust training of deep neural networks with extremely noisy labels
10. Metalabelnet: Learning to generate soft-labels from noisy-labels10.1109/TIP.2022.3183841 ↗
11. Zhou, Y., Li, X., Liu, F., Wei, Q., Chen, X., Yu, L., Xie, C., Lungren, M.P., and Xing, L. (2024, January 16–22). L2B: Learning to Bootstrap Robust Models for Combating Label Noise. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.02220 ↗
12. Algan, G., and Ulusoy, I. (2021, January 10–15). Meta soft label generation for noisy labels. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.10.1109/ICPR48806.2021.9412490 ↗
13. Li, M., Soltanolkotabi, M., and Oymak, S. (2020, January 26–28). Gradient descent with early stopping is provably robust to label noise for overparameterized neural networks. Proceedings of the International Conference on Artificial Intelligence and Statistics, Online Conference.
14. Are anchor points really indispensable in label-noise learning?
15. Ghosh, A., Kumar, H., and Sastry, P.S. (2017, January 4–9). Robust loss functions under label noise for deep neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.10894 ↗
16. Wang, X., Hua, Y., Kodirov, E., Clifton, D.A., and Robertson, N.M. (2019). IMAE for noise-robust learning: Mean absolute error does not treat examples equally and gradient magnitude’s variance matters. arXiv.
17. Generalized cross entropy loss for training deep neural networks with noisy labels
18. Wang, Y., Ma, X., Chen, Z., Luo, Y., Yi, J., and Bailey, J. (November, January 27). Symmetric cross entropy for robust learning with noisy labels. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.
19. Understanding deep learning (still) requires rethinking generalization10.1145/3446776 ↗
20. Zhang, B., Li, Y., Tu, Y., Peng, J., Wang, Y., Wu, C., Xiao, Y., and Zhao, C. (2022). Learning from noisy labels with coarse-to-fine sample credibility modeling. European Conference on Computer Vision, Springer.10.1007/978-3-031-25063-7_2 ↗
21. Li, J., Socher, R., and Hoi, S.C. (2020). Dividemix: Learning with noisy labels as semi-supervised learning. arXiv.
22. Ghosh, A., and Lan, A. (2021, January 19–25). Contrastive learning improves model robustness under label noise. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPRW53098.2021.00304 ↗
23. Karim, N., Rizve, M.N., Rahnavard, N., Mian, A., and Shah, M. (2022, January 18–24). Unicon: Combating label noise through uniform selection and contrastive learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00945 ↗
24. Li, J., Xiong, C., and Hoi, S.C. (2021, January 10–17). Learning from noisy data with robust representation learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00935 ↗
25. Tanaka, D., Ikami, D., Yamasaki, T., and Aizawa, K. (2018, January 18–23). Joint optimization framework for learning with noisy labels. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00582 ↗
26. Shen, Y., and Sanghavi, S. (2019, January 9–15). Learning with bad training data via iterative trimmed loss minimization. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA.
27. Yu, X., Han, B., Yao, J., Niu, G., Tsang, I., and Sugiyama, M. (2019, January 9–15). How does disagreement help generalization against label corruption?. Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA.
28. Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009, January 14). Curriculum learning. Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada.10.1145/1553374.1553380 ↗
29. Progressive stochastic learning for noisy labels10.1109/TNNLS.2018.2792062 ↗
30. Li, J., Wong, Y., Zhao, Q., and Kankanhalli, M.S. (2019, January 16–20). Learning to learn from noisy labeled data. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00519 ↗
31. Zhang, Z., Zhang, H., Arik, S.O., Lee, H., and Pfister, T. (2020, January 13–19). Distilling effective supervision from severe label noise. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00931 ↗
32. Zheng, G., Awadallah, A.H., and Dumais, S. (2021, January 2–9). Meta label correction for noisy label learning. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual Conference.10.1609/aaai.v35i12.17319 ↗
33. Finn, C., Abbeel, P., and Levine, S. (2017, January 6–11). Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia.
34. Reed, S., Lee, H., Anguelov, D., Szegedy, C., Erhan, D., and Rabinovich, A. (2014). Training deep neural networks on noisy labels with bootstrapping. arXiv.