Weakly Supervised Conditional Random Fields Model for Semantic Segmentation with Image Patches
Source abstract
Abstract
Image semantic segmentation (ISS) is used to segment an image into regions with differently labeled semantic category. Most of the existing ISS methods are based on fully supervised learning, which requires pixel-level labeling for training the model. As a result, it is often very time-consuming and labor-intensive, yet still subject to manual errors and subjective inconsistency. To tackle such difficulties, a weakly supervised ISS approach is proposed, in which the challenging problem of label inference from image-level to pixel-level will be particularly addressed, using image patches and conditional random fields (CRF). An improved simple linear iterative cluster (SLIC) algorithm is employed to extract superpixels. for image segmentation. Specifically, it generates various numbers of superpixels according to different images, which can be used to guide the process of image patch extraction based on the image-level labeled information. Based on the extracted image patches, the CRF model is constructed for inferring semantic class labels, which uses the potential energy function to map from the image-level to pixel-level image labels. Finally, patch based CRF (PBCRF) model is used to accomplish the weakly supervised ISS. Experiments conducted on two publicly available benchmark datasets, MSRC and PASCAL VOC 2012, have demonstrated that our proposed algorithm can yield very promising results compared to quite a few state-of-the-art ISS methods, including some deep learning-based models.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image and Video Retrieval Techniques · Advanced Neural Network Applications · Medical Image Segmentation Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030024, China
College of Information and Computer, Taiyuan University of Technology, Jinzhong 030600, China
University of Strathclyde, Department of Electronic and Electrical Engineering, Glasgow G4 0LN, UK
Funding
National Natural Science Foundation of China · Natural Science Foundation of Shanxi Province · Shanxi Provincial Key Research and Development Project
References (54)
Show source-provided references
1. Zhang, W., Zeng, S., Wang, D., and Xue, X. (2015, January 7–12). Weakly supervised semantic segmentation for social images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298888 ↗
2. Lin, G., Milan, A., Shen, C., and Reid, I. (2017, January 21–26). Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. Proceedings of the IEEE Conference CVPR, Honolulu, HI, USA.10.1109/CVPR.2017.549 ↗
3. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2014). Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv Prepr.
4. Dehghani, M., Severyn, A., Rothe, S., and Kamps, J. (2017). Learning to learn from weak supervision by full supervision. arXiv Prepr.
5. Wei, Y., Xiao, H., Shi, H., Jie, Z., Feng, J., and Huang, T.S. (2018, January 18–23). Revisiting dilated convolution: A simple approach for weakly-and semi-supervised semantic segmentation. Proceedings of the IEEE Conference CVPR, Salt Lake City, UT, USA.10.1109/CVPR.2018.00759 ↗
6. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference CVPR, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
7. Lin, D., Dai, J., Jia, J., He, K., and Sun, J. (2016, January 27–30). Scribblesup: Scribble-supervised convolutional networks for semantic segmentation. Proceedings of the IEEE Conference CVPR, Las Vegas, NV, USA.10.1109/CVPR.2016.344 ↗
8. A review on image segmentation techniques10.1016/0031-3203(93)90135-J ↗
9. Region-based segmentation and object detection
10. Edge detection techniques for image segmentation-a survey of soft computing approaches
11. Image segmentation by clustering10.1109/PROC.1979.11327 ↗
12. Al-Amri, S.S., and Kalyankar, N.V. (2010). others Image segmentation by using threshold techniques. arXiv Prepr.
13. Tang, J. (2010, January 21–24). A color image segmentation algorithm based on region growing. Proceedings of the 2010 2nd International Conference on Computer Engineering and Technology, Wuhan, China.
14. Borges, G.A., and Aldon, M.-J. (2000, January 3–7). A split-and-merge segmentation algorithm for line extraction in 2d range images. Proceedings of the 15th International Conference on Pattern Recognition (ICPR), Barcelona, Spain.
15. Kang, W., Yang, Q., and Liang, R. (2009, January 7–8). The comparative research on image segmentation algorithms. Proceedings of the 2009 First International Workshop on Education Technology and Computer Science, Wuhan, China.10.1109/ETCS.2009.417 ↗
16. Ren, X., and Malik, J. (2003, January 13–16). Learning a classification model for segmentation. Proceedings of the Ninth IEEE International Conference on Computer Vision (ICCV), Nice, France.10.1109/ICCV.2003.1238308 ↗
17. Normalized cuts and image segmentation
18. Moore, A.P., Prince, S.J., Warrell, J., Mohammed, U., and Jones, G. (2008, January 23–28). Superpixel lattices. Proceedings of the IEEE Conference CVPR, Anchorage, AK, USA.10.1109/CVPR.2008.4587471 ↗
19. Efficient graph-based image segmentation10.1023/B:VISI.0000022288.19776.77 ↗
20. Watersheds in digital spaces: An efficient algorithm based on immersion simulations10.1109/34.87344 ↗
21. Mean shift: A robust approach toward feature space analysis10.1109/34.1000236 ↗
22. SLIC superpixels compared to state-of-the-art superpixel methods10.1109/TPAMI.2012.120 ↗
23. Hsu, C., and Ding, J. (2013, January 10–13). Efficient image segmentation algorithm using SLIC superpixels and boundary-focused region merging. Proceedings of the 2013 9th International Conference on Information, Communications & Signal Processing, Tainan, Taiwan.
24. Interactive image segmentation by maximal similarity based region merging10.1016/j.patcog.2009.03.004 ↗
25. Gu, X., Deng, J.D., and Purvis, M.K. (2014, January 27–30). Improving superpixel-based image segmentation by incorporating color covariance matrix manifolds. Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), Paris, France.10.1109/ICIP.2014.7025893 ↗
26. Wei, Y., Feng, J., Liang, X., Cheng, M.-M., Zhao, Y., and Yan, S. (2017, January 21–26). Object region mining with adversarial erasing: A simple classification to semantic segmentation approach. Proceedings of the IEEE Conference CVPR, Honolulu, HI, USA.10.1109/CVPR.2017.687 ↗
27. Duygulu, P., Barnard, K., de Freitas, J.F., and Forsyth, D.A. (, January 28–31). Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. Proceedings of the European Conference on Computer Vision (ECCV), Copenhagen, Denmark, 2002.10.1007/3-540-47979-1_7 ↗
28. Zhang, K., Zhang, W., Zheng, Y., and Xue, X. (2013, January 3–9). Sparse reconstruction for weakly supervised semantic segmentation. Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence (IJCAI), Beijing, China.
29. Vezhnevets, A., Ferrari, V., and Buhmann, J. (2011, January 6–13). Weakly supervised semantic segmentation with a Multi-Image model. Proceedings of the IEEE Conference ICCV, Barcelona, Spain.10.1109/ICCV.2011.6126299 ↗
30. Liu, Y., Liu, J., Li, Z., Tang, J., and Lu, H. (2013, January 23–28). Weakly-supervised dual clustering for image semantic segmentation. Proceedings of the IEEE Conference CVPR, Portland, OR, USA.10.1109/CVPR.2013.270 ↗
31. Zhang, L., Song, M., Liu, Z., Liu, X., Bu, J., and Chen, C. (2013, January 23–28). Probabilistic graphlet cut: Exploiting spatial structure cue for weakly supervised image segmentation. Proceedings of the IEEE Conference CVPR, Portland, OR, USA.10.1109/CVPR.2013.249 ↗
32. Weakly-supervised region annotation for understanding scene images10.1007/s11042-014-2420-5 ↗
33. Noh, H., Hong, S., and Han, B. (2015, January 7–13). Learning deconvolution network for semantic segmentation. Proceedings of the IEEE Conference ICCV, Santiago, Chile.10.1109/ICCV.2015.178 ↗
34. Learning hierarchical features for scene labeling10.1109/TPAMI.2012.231 ↗
35. Qi, X., Liu, Z., Shi, J., Zhao, H., and Jia, J. (2016, January 11–14). Augmented feedback in semantic segmentation under image level supervision. Proceedings of the IEEE Conference ECCV, Amsterdam, The Netherlands.10.1007/978-3-319-46484-8_6 ↗
36. Learning to segment with image-level annotations10.1016/j.patcog.2016.01.015 ↗
37. Papandreou, G., Chen, L.-C., Murphy, K., and Yuille, A. (2015). Weakly-and semi-supervised learning of a DCNN for semantic image segmentation. arXiv Prepr.10.1109/ICCV.2015.203 ↗
38. Stc: A simple to complex framework for weakly-supervised semantic segmentation10.1109/TPAMI.2016.2636150 ↗
39. Shotton, J., Winn, J., Rother, C., and Criminisi, A. (2006, January 7–13). Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. Proceedings of the IEEE Conference ECCV, Graz, Austria.10.1007/11744023_1 ↗
40. The pascal visual object classes (voc) challenge10.1007/s11263-009-0275-4 ↗
41. The pascal visual object classes challenge: A retrospective10.1007/s11263-014-0733-5 ↗
42. Liu, M.-Y., Tuzel, O., Ramalingam, S., and Chellappa, R. (2011, January 20–25). Entropy rate superpixel segmentation. Proceedings of the IEEE Conference CVPR, Providence, RI, USA.10.1109/CVPR.2011.5995323 ↗
43. Verbeek, J., and Triggs, B. (2007, January 17–22). Region classification with markov field aspect models. Proceedings of the IEEE Conference CVPR, Minneapolis, MN, USA.10.1109/CVPR.2007.383098 ↗
44. Pathak, D., Krahenbuhl, P., and Darrell, T. (2015, January 7–13). Constrained convolutional neural networks for weakly supervised segmentation. Proceedings of the IEEE Conference ICCV, Santiago, Chile.10.1109/ICCV.2015.209 ↗
45. Pinheiro, P.O., and Collobert, R. (2015, January 7–12). From Image-Level to Pixel-Level Labeling with Convolutional Networks. Proceedings of the IEEE Conference CVPR, Boston, MA, USA.10.1109/CVPR.2015.7298780 ↗
46. Learning to exploit the prior network knowledge for weakly supervised semantic segmentation10.1109/TIP.2019.2901393 ↗
47. Adaptive fusion of color and spatial features for noise-robust retrieval of colored logo and trademark images10.1007/s11045-016-0382-7 ↗
48. Automatic image segmentation with superpixels and image-level labels10.1109/ACCESS.2019.2891941 ↗
49. A deep-learning based feature hybrid framework for spatiotemporal saliency detection inside videos10.1016/j.neucom.2018.01.076 ↗
50. Unsupervised image saliency detection with Gestalt-laws guided optimization and visual attention based refinement10.1016/j.patcog.2018.02.004 ↗
51. Background prior-based salient object detection via deep reconstruction residual
52. Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning10.1109/TGRS.2014.2374218 ↗
53. Weakly supervised deep semantic segmentation using CNN and ELM with semantic candidate regions
54. Combined multiscale segmentation convolutional neural network for rapid damage mapping from postearthquake very high-resolution images