Effect of Probabilistic Similarity Measure on Metric-Based Few-Shot Classification
Source abstract
Abstract
In developing a few-shot classification model using deep networks, the limited number of samples in each class causes difficulty in utilizing statistical characteristics of the class distributions. In this paper, we propose a method to treat this difficulty by combining a probabilistic similarity based on intra-class statistics with a metric-based few-shot classification model. Noting that the probabilistic similarity estimated from intra-class statistics and the classifier of conventional few-shot classification models have a common assumption on the class distributions, we propose to apply the probabilistic similarity to obtain loss value for episodic learning of embedding network as well as to classify unseen test data. By defining the probabilistic similarity as the probability density of difference vectors between two samples with the same class label, it is possible to obtain a more reliable estimate of the similarity especially for the case of large number of classes. Through experiments on various benchmark data, we confirm that the probabilistic similarity can improve the classification performance, especially when the number of classes is large.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Domain Adaptation and Few-Shot Learning · Anomaly Detection Techniques and Applications · Machine Learning and Data Classification
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Engineering, Kyungpook National University, Daegu 41566, Korea
References (34)
Show source-provided references
1. Backpropagation applied to handwritten zip code recognition10.1162/neco.1989.1.4.541 ↗
2. Imagenet classification with deep convolutional neural networks
3. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 8–10). Going Deeper with Convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
4. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
5. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017, January 4–9). Inception-v4, Inception-Resnet and the Impact of Residual Connections on Learning. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Franciscom, CA, USA.10.1609/aaai.v31i1.11231 ↗
6. Koch, G., Zemel, R., and Salakhutdinov, R. (2015, January 10–11). Siamese Neural Networks for One-Shot Image Recognition. Proceedings of the ICML Deep Learning Workshop, Paris, France.
7. Ravi, S., and Larochelle, H. (2017, January 24–26). Optimization as a Model for Few-Shot Learning. Proceedings of the International Conference on Learning Representations (ICLR), Toulon, France.
8. Matching networks for one shot learning
9. Snell, J., Swersky, K., and Zemel, R.S. (2021, October 01). Prototypical Networks for Few-shot Learning. NIPS. Available online: https://github.com/jakesnell/prototypical-networks.
10. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., and Hospedales, T.M. (2018, January 19–21). Learning to Compare: Relation Network for Few-Shot Learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00131 ↗
11. Garcia, V., and Bruna, J. (May, January 30). Few-Shot Learning with Graph Neural Networks. Proceedings of the 36th International Conference on Learning Representaions, Vancouver, BC, Canada.
12. Allen, K., Shelhamer, E., Shin, H., and Tenenbaum, J. (2019, January 16–20). Infinite Mixture Prototypes for Few-Shot Learning. Proceedings of the 36th Conference on Machine Learning, Long Beach, CA, USA.
13. Li, W., Wang, L., Xu, J., Huo, J., Gao, Y., and Luo, J. (2019, January 16–20). Revisiting Local Descriptor based Image-to-Class Measure for Few-Shot Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00743 ↗
14. Wertheimer, D., and Hariharan, B. (2019, January 16–20). Few-Shot Learning with Localization in Realistic Settings. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00672 ↗
15. Kim, J., Oh, T.H., Lee, S., Pan, F., and Kweon, I.S. (2019, January 16–20). Variational Prototyping-Encoder: One-Shot Learning with Prototypical Images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00969 ↗
16. Sun, Q., Liu, Y., Chua, T.S., and Schiele, B. (2019, January 16–20). Meta-Transfer Learning for Few-Shot Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00049 ↗
17. Finn, C., Abbeel, P., and Levine, S. (2017, January 6–11). Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. Proceedings of the 36th Conference on Machine Learning, Sydney, NSW, Australia.
18. Hao, F., He, F., Cheng, J., Wang, L., Cao, J., and Tao, D. (November, January 27). Collect and Select: Semantic Alignment Metric Learning for Few-Shot Learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.
19. Kaiser, Ł., Nachum, O., Roy, A., and Bengio, S. (2017, January 24–26). Learning to Remember Rare Events. Proceedings of the International Conference on Learning Representations, Toulon, France.
20. Kim, J., Lee, S., Oh, T.H., and Kweon, I.S. (2018, January 2–7). Co-Domain Embedding using Deep Quadruplet Networks for Unseen Traffic Sign Recognition. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12323 ↗
21. Chen, W.Y., Liu, Y.C., Kira, Z., Wang, Y.C.F., and Huang, J.B. (2019, January 6–9). A Closer Look at Few-Shot Classification. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA.
22. Fort, S. (2017, January 9). Gaussian Prototypical Networks for Few-Shot Learning on Omniglot. Proceedings of the NIPS 2017 Bayesian Deep Learning Workshop, Long Beach, CA, USA. Available online: https://github.com/stanislavfort/gaussian-prototypical-networks.
23. Li, W., Xu, J., Huo, J., Wang, L., Gao, Y., and Luo, J. (February, January 27). Distribution Consistency based Covariance Metric Networks for Few-Shot Learning. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
24. Liu, B., Kang, H., Li, H., Hua, G., and Vasconcelos, N. (2020, January 14–19). Few-Shot Open-Set Recognition using Meta-Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual.10.1109/CVPR42600.2020.00882 ↗
25. The mahalanobis distance
26. A New Similarity Measure Based on Intraclass Statistics for Biometrie Systems10.4218/etrij.03.0102.0017 ↗
27. Probabilistic learning of similarity measures for tensor PCA10.1016/j.patrec.2012.03.019 ↗
28. A feature analysis for dimension reduction based on a data generation model with class factors and environment factors10.1016/j.cviu.2009.06.001 ↗
29. Bayesian face recognition10.1016/S0031-3203(99)00179-X ↗
30. Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., and Lillicrap, T. (2016, January 19–24). Meta-Learning with Memory-Augmented Neural Networks. Proceedings of the 33th Conference on Machine Learning, New York, NY, USA.
31. Kingma, D.P., and Welling, M. (2014, January 14–16). Auto-Encoding Variational Bayes. Proceedings of the International Conference on Learning Representations, Banff, AB, Canada.
32. Choi, H., and Park, H. (2015, January 21–24). Measuring Similarity Between Matrix Objects for Pattern Recognition. Proceedings of the 3rd International Conference on Human-Agent Interaction, Daegu Kyungpook, Korea.10.1145/2814940.2814967 ↗
33. Lake, B., Salakhutdinov, R., Gross, J., and Tenenbaum, J. (2011, January 20–23). One Shot Learning of Simple Visual Concepts. Proceedings of the Annual Meeting of the Cognitive Science Society, Boston, MA, USA.
34. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition10.1016/j.neunet.2012.02.016 ↗