Self-Improved Learning for Salient Object Detection
Source abstract
Abstract
Salient Object Detection (SOD) aims at identifying the most visually distinctive objects in a scene. However, learning a mapping directly from a raw image to its corresponding saliency map is still challenging. First, the binary annotations of SOD impede the model from learning the mapping smoothly. Second, the annotator’s preference introduces noisy labeling in the SOD datasets. Motivated by these, we propose a novel learning framework which consists of the Self-Improvement Training (SIT) strategy and the Augmentation-based Consistent Learning (ACL) scheme. SIT aims at reducing the learning difficulty, which provides smooth labels and improves the SOD model in a momentum-updating manner. Meanwhile, ACL focuses on improving the robustness of models by regularizing the consistency between raw images and their corresponding augmented images. Extensive experiments on five challenging benchmark datasets demonstrate that the proposed framework can play a plug-and-play role in various existing state-of-the-art SOD methods and improve their performances on multiple benchmarks without any architecture modification.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Visual Attention and Saliency Detection · Advanced Image and Video Retrieval Techniques · Olfactory and Sensory Function Studies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
References (65)
Show source-provided references
1. Mahadevan, V., and Vasconcelos, N. (2009, January 20–25). Saliency-based discriminant tracking. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition (CVPR), Miami Beach, FL, USA.10.1109/CVPR.2009.5206573 ↗
2. Fang, H., Gupta, S., Iandola, F.N., Srivastava, R.K., Deng, L., Dollár, P., Gao, J., He, X., Mitchell, M., and Platt, J.C. (2015, January 7–12). From captions to visual concepts and back. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298754 ↗
3. Study of Saliency in Objective Video Quality Assessment10.1109/TIP.2017.2651410 ↗
4. Zhao, R., Ouyang, W., and Wang, X. (2013, January 25–27). Unsupervised Salience Learning for Person Re-identification. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA.10.1109/CVPR.2013.460 ↗
5. Liu, G., and Fan, D. (2013, January 7–8). A Model of Visual Attention for Natural Image Retrieval Computing Companion. Proceedings of the International Conference on Information Science and Cloud, Guangzhou, China.10.1109/ISCC-C.2013.21 ↗
6. Self-Erasing Network for Integral Object Attention
7. Leveraging Prior-Knowledge for Weakly Supervised Object Detection under a Collaborative Self-Paced Curriculum Learning Framework10.1007/s11263-018-1112-4 ↗
8. Li, Y., Hou, X., Koch, C., Rehg, J.M., and Yuille, A.L. (2014, January 24–27). The Secrets of Salient Object Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.43 ↗
9. Yang, C., Zhang, L., Lu, H., Ruan, X., and Yang, M.H. (2013, January 25–27). Saliency Detection via Graph-Based Manifold Ranking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA.10.1109/CVPR.2013.407 ↗
10. Yan, Q., Xu, L., Shi, J., and Jia, J. (2013, January 25–27). Hierarchical Saliency Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA.10.1109/CVPR.2013.153 ↗
11. Li, G., and Yu, Y. (2015, January 7–12). Visual Saliency Based on Multiscale Deep Features. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.
12. Wang, L., Lu, H., Ruan, X., and Yang, M. (2015, January 7–12). Deep networks for saliency detection via local estimation and global search. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298938 ↗
13. Zhao, R., Ouyang, W., Li, H., and Wang, X. (2015, January 7–12). Saliency detection by multi-context deep learning. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298731 ↗
14. Zhang, J., Sclaroff, S., Lin, Z., Shen, X., Price, B.L., and Mech, R. (2016, January 27–30). Unconstrained Salient Object Detection via Proposal Subset Optimization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.618 ↗
15. Lee, G., Tai, Y., and Kim, J. (2016, January 27–30). Deep Saliency with Encoded Low Level Distance Map and High Level Features. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.78 ↗
16. SuperCNN: A Superpixelwise Convolutional Neural Network for Salient Object Detection10.1007/s11263-015-0822-0 ↗
17. Kim, J., and Pavlovic, V. (2016, January 11–14). A Shape-Based Approach for Salient Object Detection Using Deep Learning. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.10.1007/978-3-319-46493-0_28 ↗
18. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully Convolutional Networks for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
19. Fan, D.P., Cheng, M.M., Liu, J.J., Gao, S.H., Hou, Q., and Borji, A. (2018, January 8–14). Salient Objects in Clutter: Bringing Salient Object Detection to the Foreground. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01267-0_12 ↗
20. Li, G., Xie, Y., Lin, L., and Yu, Y. (2017, January 21–26). Instance-Level Salient Object Segmentation. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.34 ↗
21. Zhang, P., Wang, D., Lu, H., Wang, H., and Yin, B. (2017, January 22–29). Learning Uncertain Convolutional Features for Accurate Saliency Detection. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.32 ↗
22. Wang, T., Zhang, L., Wang, S., Lu, H., Yang, G., Ruan, X., and Borji, A. (2018, January 18–23). Detect Globally, Refine Locally: A Novel Approach to Saliency Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00330 ↗
23. Liu, N., Han, J., and Yang, M.H. (2018, January 18–23). PiCANet: Learning Pixel-Wise Contextual Attention for Saliency Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00326 ↗
24. Zhang, L., Dai, J., Lu, H., He, Y., and Wang, G. (2018, January 18–23). A Bi-Directional Message Passing Model for Salient Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00187 ↗
25. Context-Aware Graph Label Propagation Network for Saliency Detection10.1109/TIP.2020.3002083 ↗
26. Liu, Z., Wang, Y., Tu, Z., Xiao, Y., and Tang, B. (2021, January 20–24). TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Network. Proceedings of the ACM International Conference on Multimedia, Chengdu, China.10.1145/3474085.3475601 ↗
27. Wang, X., Jiang, B., Wang, X., and Luo, B. (2021). MTFNet: Mutual-Transformer Fusion Network for RGB-D Salient Object Detection. arXiv.
28. Mao, Y., Zhang, J., Wan, Z., Dai, Y., Li, A., Lv, Y., Tian, X., Fan, D., and Barnes, N. (2021). Transformer Transforms Salient Object Detection and Camouflaged Object Detection. arXiv.
29. Qiu, Y., Liu, Y., Zhang, L., and Xu, J. (2021). Boosting Salient Object Detection with Transformer-based Asymmetric Bilateral U-Net. arXiv.
30. Liu, N., Zhang, N., Wan, K., Shao, L., and Han, J. (2021, January 11–17). Visual Saliency Transformer. Proceedings of the International Conference on Computer Vision (ICCV), Montreal, BC, Canada.10.1109/ICCV48922.2021.00468 ↗
31. Zhang, X., Wang, T., Qi, J., Lu, H., and Wang, G. (2018, January 18–23). Progressive Attention Guided Recurrent Network for Salient Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00081 ↗
32. Wang, W., Zhao, S., Shen, J., Hoi, S.C.H., and Borji, A. (2019, January 15–20). Salient Object Detection With Pyramid Attention and Salient Edges. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00154 ↗
33. Feng, M., Lu, H., and Ding, E. (2019, January 15–20). Attentive Feedback Network for Boundary-Aware Salient Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00172 ↗
34. Zhao, X., Pang, Y., Zhang, L., Lu, H., and Zhang, L. (2020, January 23–28). Suppress and Balance: A Simple Gated Network for Salient Object Detection. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK.10.1007/978-3-030-58536-5_3 ↗
35. Deeply Supervised Salient Object Detection with Short Connections10.1109/TPAMI.2018.2815688 ↗
36. Zhang, P., Wang, D., Lu, H., Wang, H., and Ruan, X. (2017, January 22–29). Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.31 ↗
37. Wang, Y., Wang, R., Fan, X., Wang, T., and He, X. (2023, January 18–22). Pixels, Regions, and Objects: Multiple Enhancement for Salient Object Detection. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada.10.1109/CVPR52729.2023.00967 ↗
38. Zhao, J., Liu, J., Fan, D., Cao, Y., Yang, J., and Cheng, M. (November, January 27). EGNet: Edge Guidance Network for Salient Object Detection. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Republic of Korea.
39. Li, X., Yang, F., Cheng, H., Liu, W., and Shen, D. (2018, January 8–14). Contour Knowledge Transfer for Salient Object Detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01267-0_22 ↗
40. Qin, X., Zhang, Z., Huang, C., Gao, C., Dehghan, M., and Jagersand, M. (2019, January 15–20). BASNet: Boundary-Aware Salient Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00766 ↗
41. Contour-Aware Loss: Boundary-Aware Learning for Salient Object Segmentation10.1109/TIP.2020.3037536 ↗
42. Wang, L., Lu, H., Wang, Y., Feng, M., Wang, D., Yin, B., and Ruan, X. (, January 21–26). Learning to Detect Salient Objects with Image-Level Supervision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.
43. Li, G., Xie, Y., and Lin, L. (2018, January 2–7). Weakly Supervised Salient Object Detection Using Image Labels. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12308 ↗
44. Zeng, Y., Zhuge, Y., Lu, H., Zhang, L., Qian, M., and Yu, Y. (2019, January 15–20). Multi-source weak supervision for saliency detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00623 ↗
45. Zhang, J., Xie, J., and Barnes, N. (2020, January 23–28). Learning Noise-Aware Encoder-Decoder from Noisy Labels by Alternating back-propagation for saliency detection. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK.10.1007/978-3-030-58520-4_21 ↗
46. Piao, Y., Wang, J., Zhang, M., and Lu, H. (2021, January 19–25). MFNet: Multi-filter directive network for weakly supervised salient object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Virtual.10.1109/ICCV48922.2021.00410 ↗
47. Zhang, J., Yu, X., Li, A., Song, P., Liu, B., and Dai, Y. (, January 13–19). Weakly-Supervised Salient Object Detection via Scribble Annotations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.
48. Yu, S., Zhang, B., Xiao, J., and Lim, E.G. (2021, January 4–7). Structure-consistent weakly supervised salient object detection with local saliency coherence. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.10.1609/aaai.v35i4.16434 ↗
49. Gao, S., Zhang, W., Wang, Y., Guo, Q., Zhang, C., He, Y., and Zhang, W. (March, January 22). Weakly-Supervised Salient Object Detection Using Point Supervision. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.
50. Wu, Z., Wang, L., Wang, W., Xia, Q., Chen, C., Hao, A., and Li, S. (2023, January 7–14). Pixel is All You Need: Adversarial Trajectory-Ensemble Active Learning for Salient Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i3.25390 ↗
51. Activation to saliency: Forming high-quality labels for unsupervised salient object detection10.1109/TCSVT.2022.3203595 ↗
52. Zhou, H., Qiao, B., Yang, L., Lai, J., and Xie, X. (2023, January 13–19). Texture-Guided Saliency Distilling for Unsupervised Salient Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52729.2023.00701 ↗
53. Song, Y., Gao, S., Xing, H., Cheng, Y., Wang, Y., and Zhang, W. (November, January 29). Towards End-to-End Unsupervised Saliency Detection with Self-Supervised Top-Down Context. Proceedings of the ACM International Conference on Multimedia, Ottawa, Canada.
54. Wu, Z., Su, L., and Huang, Q. (2019, January 15–20). Cascaded Partial Decoder for Fast and Accurate Salient Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00403 ↗
55. Fan, D., Cheng, M., Liu, Y., Li, T., and Borji, A. (2017, January 22–29). Structure-Measure: A New Way to Evaluate Foreground Maps. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.487 ↗
56. Fan, D., Gong, C., Cao, Y., Ren, B., Cheng, M., and Borji, A. (2018, January 13–19). Enhanced-alignment Measure for Binary Foreground Map Evaluation. Proceedings of the International Joint Conference on Artificial Intelligence, Stockholm, Sweden.10.24963/ijcai.2018/97 ↗
57. Wei, J., Wang, S., and Huang, Q. (2020, January 7–12). F3Net: Fusion, Feedback and Focus for Salient Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.
58. Pang, Y., Zhao, X., Zhang, L., and Lu, H. (2020, January 13–19). Multi-Scale Interactive Network for Salient Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00943 ↗
59. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
60. DeVries, T., and Taylor, G.W. (2017). Improved Regularization of Convolutional Neural Networks with Cutout. arXiv.
61. Deng, Z., Hu, X., Zhu, L., Xu, X., Qin, J., Han, G., and Heng, P. (2018, January 13–19). R3Net: Recurrent Residual Refinement Network for Saliency Detection. Proceedings of the International Joint Conference on Artificial Intelligence, Stockholm, Sweden.10.24963/ijcai.2018/95 ↗
62. Wang, W., Shen, J., Cheng, M., and Shao, L. (2019, January 15–20). An Iterative and Cooperative Top-Down and Bottom-Up Inference Network for Salient Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00612 ↗
63. Liu, J., Hou, Q., Cheng, M., Feng, J., and Jiang, J. (2019, January 15–20). A Simple Pooling-Based Design for Real-Time Salient Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00404 ↗
64. Chen, S., Tan, X., Wang, B., and Hu, X. (2018, January 8–14). Reverse Attention for Salient Object Detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01240-3_15 ↗
65. Laine, S., and Aila, T. (2017, January 24–26). Temporal Ensembling for Semi-Supervised Learning. Proceedings of the International Conference on Learning Representations, Toulon, France.