A Multi-View Integrated Ensemble for the Background Discrimination of Semi-Supervised Semantic Segmentation
Source abstract
Abstract
The key to semi-supervised semantic segmentation is to assign the appropriate pseudo-label to the pixels of unlabeled images. Recently, various approaches to consistency-based training and the filtering of reliable pseudo-labels have shown remarkable results. Nonetheless, there are still issues to be addressed. We find that recent approaches have specific problems in common. In pseudo-labels for training unlabeled images, we confirm that false foreground class pseudo-labels are mostly caused by background class confusion, not confusion between different foreground classes. To solve this problem, we propose a foreground and background discrimination model for semi-supervised semantic segmentation. Our proposed model is trained using a novel approach called multi-view integrated ensemble (MVIE) via output perturbation. Experimental results in various partition protocols show that our approach outperforms the existing state of the art (SOTA) in binary prediction on unlabeled data, and the segmentation model trained with the help of our model outperforms existing models.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Advanced Image and Video Retrieval Techniques · Domain Adaptation and Few-Shot Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Applied Statistics, Konkuk University, Seoul 05029, Republic of Korea
AI Analytics Team, Mustree, Seoul 05029, Republic of Korea
Funding
Konkuk University Researcher · National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT)
References (38)
Show source-provided references
1. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
2. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
3. Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022, January 18–24). A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01167 ↗
4. Yuan, Y., Chen, X., Chen, X., and Wang, J. (2019). Segmentation transformer: Object-contextual representations for semantic segmentation. arXiv.10.1007/978-3-030-58539-6_11 ↗
5. Cho, J., Lee, K., Shin, E., Choy, G., and Do, S. (2015). How much data is needed to train a medical image deep learning system to achieve necessary high accuracy?. arXiv.
6. Big data deep learning: Challenges and perspectives10.1109/ACCESS.2014.2325029 ↗
7. Wang, Y., Wang, H., Shen, Y., Fei, J., Li, W., Jin, G., Wu, L., Zhao, R., and Le, X. (2022, January 18–24). Semi-supervised semantic segmentation using unreliable pseudo-labels. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00421 ↗
8. Liu, Y., Tian, Y., Chen, Y., Liu, F., Belagiannis, V., and Carneiro, G. (2022, January 18–24). Perturbed and strict mean teachers for semi-supervised semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00422 ↗
9. Chen, X., Yuan, Y., Zeng, G., and Wang, J. (2021, January 20–25). Semi-supervised semantic segmentation with cross pseudo supervision. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00264 ↗
10. Conservative-progressive collaborative learning for semi-supervised semantic segmentation10.1109/TIP.2023.3242819 ↗
11. Lee, D.H. (2013, January 16–21). Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. Proceedings of the Workshop on Challenges in Representation Learning, ICML, Atlanta, GA, USA.
12. Arazo, E., Ortego, D., Albert, P., O’Connor, N.E., and McGuinness, K. (2020, January 19–24). Pseudo-labeling and confirmation bias in deep semi-supervised learning. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.10.1109/IJCNN48605.2020.9207304 ↗
13. Pham, H., Dai, Z., Xie, Q., and Le, Q.V. (2021, January 20–25). Meta pseudo labels. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01139 ↗
14. Fixmatch: Simplifying semi-supervised learning with consistency and confidence
15. Ouali, Y., Hudelot, C., and Tami, M. (2020, January 13–19). Semi-supervised semantic segmentation with cross-consistency training. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01269 ↗
16. Tu, P., Huang, Y., Ji, R., Zheng, F., and Shao, L. (2021). Guidedmix-net: Learning to improve pseudo masks using labeled images as reference. arXiv.
17. DeVries, T., and Taylor, G.W. (2017). Improved regularization of convolutional neural networks with cutout. arXiv.
18. Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., and Yoo, Y. (November, January 27). Cutmix: Regularization strategy to train strong classifiers with localizable features. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
19. A survey on semi-supervised learning10.1007/s10994-019-05855-6 ↗
20. The pascal visual object classes challenge: A retrospective10.1007/s11263-014-0733-5 ↗
21. Rethinking pre-training and self-training
22. Hung, W.C., Tsai, Y.H., Liou, Y.T., Lin, Y.Y., and Yang, M.H. (2018). Adversarial learning for semi-supervised semantic segmentation. arXiv.
23. Semi-supervised semantic segmentation with high-and low-level consistency10.1109/TPAMI.2019.2960224 ↗
24. Binary-classifiers-enabled filters for semi-supervised learning10.1109/ACCESS.2021.3124200 ↗
25. Semi-supervised salient object detection using a linear feedback control system model10.1109/TCYB.2018.2793278 ↗
26. Liu, J., Zhang, J., and Barnes, N. (2021). Semi-supervised salient object detection with effective confidence estimation. arXiv.10.1109/CVPR46437.2021.01655 ↗
27. Semi-supervised active salient object detection10.1016/j.patcog.2021.108364 ↗
28. Deep high-resolution representation learning for visual recognition10.1109/TPAMI.2020.2983686 ↗
29. Wang, W., Zhou, T., Yu, F., Dai, J., Konukoglu, E., and Van Gool, L. (2021, January 11–17). Exploring cross-image pixel contrast for semantic segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00721 ↗
30. Zhou, T., Wang, W., Konukoglu, E., and Van Gool, L. (2022, January 18–24). Rethinking semantic segmentation: A prototype view. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00261 ↗
31. Yang, L., Qi, L., Feng, L., Zhang, W., and Shi, Y. (2023, January 18–22). Revisiting weak-to-strong consistency in semi-supervised semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00699 ↗
32. Kimhi, M., Kimhi, S., Zheltonozhskii, E., Litany, O., and Baskin, C. (2023). Semi-Supervised Semantic Segmentation via Marginal Contextual Information. arXiv.
33. Hariharan, B., Arbeláez, P., Bourdev, L., Maji, S., and Malik, J. (2011, January 6–13). Semantic contours from inverse detectors. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.10.1109/ICCV.2011.6126343 ↗
34. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2014). Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv.
35. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
36. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
37. Ioffe, S., and Szegedy, C. (2015, January 6–11). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning, PMLR, Lille, France.
38. Pytorch: An imperative style, high-performance deep learning library