Mask Mixup Model: Enhanced Contrastive Learning for Few-Shot Learning
Source abstract
Abstract
Few-shot image classification aims to improve the performance of traditional image classification when faced with limited data. Its main challenge lies in effectively utilizing sparse sample label data to accurately predict the true feature distribution. Recent approaches have employed data augmentation techniques like random Mask or mixture interpolation to enhance the diversity and generalization of labeled samples. However, these methods still encounter several issues: (1) random Mask can lead to complete blockage or exposure of foreground, causing loss of crucial sample information; and (2) uniform data distribution after mixture interpolation makes it difficult for the model to differentiate between different categories and effectively distinguish their boundaries. To address these challenges, this paper introduces a novel data augmentation method based on saliency mask blending. Firstly, it selectively preserves key image features through adaptive selection and retention using visual feature occlusion fusion and confidence clipping strategies. Secondly, a visual feature saliency fusion approach is employed to calculate the importance of various image regions, guiding the blending process to produce more diverse and enriched images with clearer category boundaries. The proposed method achieves outstanding performance on multiple standard few-shot image classification datasets (miniImageNet, tieredImageNet, Few-shot FC100, and CUB), surpassing state-of-the-art methods by approximately 0.2–1%.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Domain Adaptation and Few-Shot Learning · Multimodal Machine Learning Applications · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Laboratory of Computer Network and Information Integration, School of Computer Science and Engineering, Southeast University, Nanjing 211189, China
Institute of NR Electric Co., Ltd., Nanjing 211102, China
School of Computer Science, Nanjing University of Information Science and Technology, Nanjing 210044, China
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
Funding
National Natural Science Foundation of China
References (33)
Show source-provided references
1. Chen, Y., Liu, Z., Xu, H., Darrell, T., and Wang, X. (2021, January 10–17). Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00893 ↗
2. Padmanabhan, D., Gowda, S., Arani, E., and Zonooz, B. (2023, January 17–24). LSFSL: Leveraging Shape Information in Few-shot Learning. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00525 ↗
3. Qiao, Q., Xie, Y., Zeng, Z., and Li, F. (2023). TALDS-Net: Task-Aware Adaptive Local Descriptors Selection for Few-shot Image Classification. arXiv.10.1109/ICASSP48485.2024.10448167 ↗
4. Prototypical networks for few-shot learning
5. Deepemd: Differentiable earth mover’s distance for few-shot learning
6. Chen, H., Li, H., Li, Y., and Chen, C. (2022, January 6–9). Multi-level metric learning for few-shot image recognition. Proceedings of the International Conference on Artificial Neural Networks, Bristol, UK.10.1007/978-3-031-15919-0_21 ↗
7. Mangla, P., Kumari, N., Sinha, A., Singh, M., Krishnamurthy, B., and Balasubramanian, V.N. (2020, January 1–5). Charting the right manifold: Manifold mixup for few-shot learning. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA.10.1109/WACV45572.2020.9093338 ↗
8. Liu, C., Fu, Y., Xu, C., Yang, S., Li, J., Wang, C., and Zhang, L. (2021, January 2–9). Learning a few-shot embedding model with contrastive learning. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.10.1609/aaai.v35i10.17047 ↗
9. Zhuo, L., Fu, Y., Chen, J., Cao, Y., and Jiang, Y.G. (2022, January 10–14). Tgdm: Target guided dynamic mixup for cross-domain few-shot learning. Proceedings of the 30th ACM International Conference on Multimedia, Lisboa, Portugal.10.1145/3503161.3548052 ↗
10. Bachman, P., Alsharif, O., and Precup, D. (2014). Learning with pseudo-ensembles. Adv. Neural Inf. Process. Syst., 27.
11. Kang, D., Kwon, H., Min, J., and Cho, M. (2021, January 11–17). Relational embedding for few-shot classification. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00870 ↗
12. Yang, Z., Wang, J., and Zhu, Y. (2022, January 6–9). Few-shot classification with contrastive learning. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-031-20044-1_17 ↗
13. Hou, R., Chang, H., Ma, B., Shan, S., and Chen, X. (2019). Cross attention network for few-shot classification. Adv. Neural Inf. Process. Syst., 32.
14. Xie, J., Long, F., Lv, J., Wang, Q., and Li, P. (2022, January 18–24). Joint distribution matters: Deep brownian distance covariance for few-shot classification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00781 ↗
15. Xu, C., Fu, Y., Liu, C., Wang, C., Li, J., Huang, F., Zhang, L., and Xue, X. (2021, January 20–25). Learning dynamic alignment via meta-filter for few-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00514 ↗
16. Wertheimer, D., Tang, L., and Hariharan, B. (2021, January 20–25). Few-shot classification with feature map reconstruction networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00792 ↗
17. Fei, N., Gao, Y., Lu, Z., and Xiang, T. (2021, January 11–17). Z-score normalization, hubness, and few-shot learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00021 ↗
18. Finn, C., Abbeel, P., and Levine, S. (2017, January 6–11). Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the International Conference on Machine Learning. PMLR, Sydney, Australia.
19. Oreshkin, B., Rodríguez López, P., and Lacoste, A. (2018). Tadam: Task dependent adaptive metric for improved few-shot learning. Adv. Neural Inf. Process. Syst., 31.
20. Zhang, X., Meng, D., Gouk, H., and Hospedales, T.M. (2021, January 11–17). Shallow bayesian meta learning for real-world few-shot recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00069 ↗
21. Tian, Y., Wang, Y., Krishnan, D., Tenenbaum, J.B., and Isola, P. (2020, January 23–28). Rethinking few-shot image classification: A good embedding is all you need?. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part XIV 16.10.1007/978-3-030-58568-6_16 ↗
22. Dvornik, N., Schmid, C., and Mairal, J. (November, January 27). Diversity with cooperation: Ensemble methods for few-shot classification. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
23. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., and Hospedales, T.M. (2018, January 18–22). Learning to compare: Relation network for few-shot learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00131 ↗
24. Chen, Z., Fu, Y., Wang, Y.X., Ma, L., Liu, W., and Hebert, M. (2019, January 15–20). Image deformation meta-networks for one-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00888 ↗
25. Yang, S., Liu, L., and Xu, M. (2021). Free lunch for few-shot learning: Distribution calibration. arXiv.
26. Afrasiyabi, A., Lalonde, J.F., and Gagné, C. (2020, January 23–28). Associative alignment for few-shot image classification. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part V 16.10.1007/978-3-030-58558-7_2 ↗
27. Afrasiyabi, A., Lalonde, J.F., and Gagné, C. (2021, January 11–17). Mixture-based feature space learning for few-shot image classification. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00891 ↗
28. Ziko, I., Dolz, J., Granger, E., and Ayed, I.B. (2020, January 13–18). Laplacian regularized few-shot learning. Proceedings of the International Conference on Machine Learning. PMLR, Virtual.
29. Afrasiyabi, A., Larochelle, H., Lalonde, J.F., and Gagné, C. (2022, January 18–24). Matching feature sets for few-shot image classification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00881 ↗
30. Vinyals, O., Blundell, C., Lillicrap, T., and Wierstra, D. (2016). Matching networks for one shot learning. Adv. Neural Inf. Process. Syst., 29.
31. Ren, M., Triantafillou, E., Ravi, S., Snell, J., Swersky, K., Tenenbaum, J.B., Larochelle, H., and Zemel, R.S. (2018). Meta-learning for semi-supervised few-shot classification. arXiv.
32. Bertinetto, L., Henriques, J.F., Torr, P.H., and Vedaldi, A. (2018). Meta-learning with differentiable closed-form solvers. arXiv.
33. Chen, C., Li, O., Tao, D., Barnett, A., Rudin, C., and Su, J.K. (2019). This looks like that: Deep learning for interpretable image recognition. Adv. Neural Inf. Process. Syst., 32.10.1609/hcomp.v7i1.5265 ↗