On the Importance of Diversity When Training Deep Learning Segmentation Models with Error-Prone Pseudo-Labels
Source abstract
Abstract
The key to training deep learning (DL) segmentation models lies in the collection of annotated data. The annotation process is, however, generally expensive in human resources. Our paper leverages deep or traditional machine learning methods trained on a small set of manually labeled data to automatically generate pseudo-labels on large datasets, which are then used to train so-called data-reinforced deep learning models. The relevance of the approach is demonstrated in two applicative scenarios that are distinct both in terms of task and pseudo-label generation procedures, enlarging the scope of the outcomes of our study. Our experiments reveal that (i) data reinforcement helps, even with error-prone pseudo-labels, (ii) convolutional neural networks have the capability to regularize their training with respect to labeling errors, and (iii) there is an advantage to increasing diversity when generating the pseudo-labels, either by enriching the manual annotation through accurate annotation of singular samples, or by considering soft pseudo-labels per sample when prior information is available about their certainty.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning and Data Classification · Advanced Neural Network Applications · Domain Adaptation and Few-Shot Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
ICTEAM Institute, UCLouvain, 1348 Louvain-la-Neuve, Belgium
ELI Institute, UCLouvain, 1348 Louvain-la-Neuve, Belgium
Funding
China Scholarship Council · Belgian F.N.R.S
References (54)
Show source-provided references
1. Visualizing the effects of predictor variables in black box supervised learning models10.1111/rssb.12377 ↗
2. Supervised learning: Classification
3. Application of supervised learning to validation of damage detection10.1007/s00419-020-01779-z ↗
4. Zhou, Z.H. (2021). Semi-supervised learning. Machine Learning, Springer.10.1007/978-981-15-1967-3_13 ↗
5. Ouali, Y., Hudelot, C., and Tami, M. (2020). An overview of deep semi-supervised learning. arXiv.
6. Zheng, M., You, S., Huang, L., Wang, F., Qian, C., and Xu, C. (2022, January 18–24). SimMatch: Semi-supervised Learning with Similarity Matching. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01407 ↗
7. Sayez, N., and De Vleeschouwer, C. (2022, January 21–25). Accelerating the creation of instance segmentation training sets through bounding box annotation. Proceedings of the 2022 26th International Conference on Pattern Recognition (ICPR), Montreal, QC, Canada.10.1109/ICPR56361.2022.9956321 ↗
8. Mendel, R., De Souza, L.A., Rauber, D., Papa, J.P., and Palm, C. (2020, January 23–28). Semi-supervised segmentation based on error-correcting supervision. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part XXIX 16.10.1007/978-3-030-58526-6_9 ↗
9. Xie, Q., Luong, M.T., Hovy, E., and Le, Q.V. (2020, January 13–19). Self-training with noisy student improves imagenet classification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01070 ↗
10. Arazo, E., Ortego, D., Albert, P., O’Connor, N.E., and McGuinness, K. (2020, January 19–24). Pseudo-labeling and confirmation bias in deep semi-supervised learning. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.10.1109/IJCNN48605.2020.9207304 ↗
11. Yang, L., Zhuo, W., Qi, L., Shi, Y., and Gao, Y. (2022, January 18–24). St++: Make self-training work better for semi-supervised semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00423 ↗
12. Improving semantic segmentation via efficient self-training10.1109/TPAMI.2021.3138337 ↗
13. Image segmentation scale parameter optimization and land cover classification using the Random Forest algorithm10.1080/14498596.2010.487851 ↗
14. A comparative study of automated legal text classification using random forests and deep learning10.1016/j.ipm.2021.102798 ↗
15. Fröhlich, B., Rodner, E., and Denzler, J. (2012, January 5–9). Semantic segmentation with millions of features: Integrating multiple cues in a combined random forest approach. Proceedings of the Asian Conference on Computer Vision, Daejeon, Republic of Korea.10.1007/978-3-642-37331-2_17 ↗
16. Random Ferns for Semantic Segmentation of PolSAR Images
17. Review the state-of-the-art technologies of semantic segmentation based on deep learning10.1016/j.neucom.2022.01.005 ↗
18. Xiao, Z., Liu, B., Geng, L., Zhang, F., and Liu, Y. (2020). Segmentation of lung nodules using improved 3D-UNet neural network. Symmetry, 12.10.3390/sym12111787 ↗
19. Rizve, M.N., Duarte, K., Rawat, Y.S., and Shah, M. (2021). In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. arXiv.
20. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling
21. Wei, C., Shen, K., Chen, Y., and Ma, T. (2020). Theoretical analysis of self-training with deep networks on unlabeled data. arXiv.
22. Liu, J., Yao, J., Bagheri, M., Sandfort, V., and Summers, R.M. (2019, January 8–11). A semi-supervised CNN learning method with pseudo-class labels for atherosclerotic vascular calcification detection. Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy.10.1109/ISBI.2019.8759189 ↗
23. A semi-supervised convolutional neural network for hyperspectral image classification10.1080/2150704X.2017.1331053 ↗
24. Pintelas, E., Livieris, I.E., and Pintelas, P. (2020). A grey-box ensemble model exploiting black-box accuracy and white-box intrinsic interpretability. Algorithms, 13.10.3390/a13010017 ↗
25. Semantic image segmentation using scant pixel annotations10.3390/make4030029 ↗
26. Zou, Y., Zhang, Z., Zhang, H., Li, C.L., Bian, X., Huang, J.B., and Pfister, T. (2020). Pseudoseg: Designing pseudo labels for semantic segmentation. arXiv.
27. Semi-supervised semantic segmentation with prototype-based consistency regularization
28. Semi-supervised deep learning via transformation consistency regularization for remote sensing image semantic segmentation10.1109/JSTARS.2022.3203750 ↗
29. Vu, T.H., Jain, H., Bucher, M., Cord, M., and Pérez, P. (2019, January 15–20). Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00262 ↗
30. Adapt everywhere: Unsupervised adaptation of point-clouds and entropy minimization for multi-modal cardiac image segmentation10.1109/TMI.2021.3066683 ↗
31. Zeng, G., Peng, H., Li, A., Liu, Z., Liu, C., Yu, P.S., and He, L. (2023). Unsupervised Skin Lesion Segmentation via Structural Entropy Minimization on Multi-Scale Superpixel Graphs. arXiv.10.1109/ICDM58522.2023.00086 ↗
32. Cioppa, A., Deliege, A., Istasse, M., De Vleeschouwer, C., and Van Droogenbroeck, M. (2019, January 15–20). ARTHuS: Adaptive real-time human segmentation in sports through online distillation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.10.1109/CVPRW.2019.00306 ↗
33. Baldeon Calisto, M. (2022). Teacher-student semi-supervised approach for medical image segmentation. MICCAI Challenge on Fast and Low-Resource Semi-supervised Abdominal Organ Segmentation, Springer.10.1007/978-3-031-23911-3_14 ↗
34. Ilastik: Interactive machine learning for (bio) image analysis10.1038/s41592-019-0582-9 ↗
35. RootPainter: Deep learning segmentation of biological images with corrective annotation10.1111/nph.18387 ↗
36. Si, S., Zhang, H., Keerthi, S.S., Mahajan, D., Dhillon, I.S., and Hsieh, C.J. (2017, January 6–11). Gradient boosted decision trees for high dimensional sparse output. Proceedings of the 34th International Conference on Machine Learning, PMLR, Sydney, NSW, Australia.
37. The random forest algorithm for statistical learning10.1177/1536867X20909688 ↗
38. Browet, A., De Vleeschouwer, C., Jacques, L., Mathiah, N., Saykali, B., and Migeotte, I. (2016, January 25–28). Cell segmentation with random ferns and graph-cuts. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.10.1109/ICIP.2016.7533140 ↗
39. Scene-specific classifier for effective and efficient team sport players detection from a single calibrated camera10.1016/j.cviu.2017.01.001 ↗
40. Bay, Y.Y., and Yearick, K.A. (2024). Machine Learning vs. Deep Learning: The Generalization Problem. arXiv.
41. Neyshabur, B., Bhojanapalli, S., McAllester, D., and Srebro, N. (2017, January 4–9). Exploring generalization in deep learning. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA.
42. Kawaguchi, K., Kaelbling, L.P., and Bengio, Y. (2022). Generalization in Deep Learning. Mathematical Aspects of Deep Learning, Cambridge University Press.10.1017/9781009025096.003 ↗
43. Léger, J., Leyssens, L., Kerckhofs, G., and De Vleeschouwer, C. (2022). Ensemble learning and test-time augmentation for the segmentation of mineralized cartilage versus bone in high-resolution microCT images. Comput. Biol. Med., 148.10.1016/j.compbiomed.2022.105932 ↗
44. Gal, Y. (2024, June 08). Uncertainty in Deep Learning. Available online: http://106.54.215.74/2019/20190729-liuzy.pdf.
45. Smith, L., and Gal, Y. (2018). Understanding measures of uncertainty for adversarial example detection. arXiv.
46. Cytomine: Toward an open and collaborative software platform for digital pathology bridged to molecular investigations10.1002/prca.201800057 ↗
47. Li, Y., Zhang, J., Gao, P., Jiang, L., and Chen, M. (2018, January 27–29). Grab cut image segmentation based on image region. Proceedings of the 2018 IEEE 3rd international conference on image, vision and computing (ICIVC), Chongqing, China.10.1109/ICIVC.2018.8492818 ↗
48. Fast keypoint recognition using random ferns10.1109/TPAMI.2009.23 ↗
49. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany. Proceedings, Part III 18.
50. MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation10.1016/j.neunet.2019.08.025 ↗
51. Yang, N., Joos, V., Jacquemart, A.L., Buyens, C., and De Vleeschouwer, C. (2022, January 18–24). Using Pure Pollen Species When Training a CNN To Segment Pollen Mixtures. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, New Orleans, LA, USA.10.1109/CVPRW56347.2022.00176 ↗
52. Jadon, S. (2020, January 27–29). A survey of loss functions for semantic segmentation. Proceedings of the 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), Via del Mar, Chile.10.1109/CIBCB48159.2020.9277638 ↗
53. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019, January 8–14). Pytorch: An imperative style, high-performance deep learning library. Proceedings of the Advances in Neural Information Processing Systems 32 (NeurIPS 2019), Vancouver, BC, Canada.
54. Understanding deep learning (still) requires rethinking generalization10.1145/3446776 ↗