Multi-Head Attention Graph Network for Few Shot Learning
Source abstract
Abstract
The majority of existing graph-network-based few-shot models focus on a node-similarity update mode. The lack of adequate information intensifies the risk of overtraining. In this paper, we propose a novel Multi-head Attention Graph Network to excavate discriminative relation and fulfill effective information propagation. For edge update, the node-level attention is used to evaluate the similarities between the two nodes and the distribution-level attention extracts more in-deep global relation. The cooperation between those two parts provides a discriminative and comprehensive expression for edge feature. For node update, we embrace the label-level attention to soften the noise of irrelevant nodes and optimize the update direction. Our proposed model is verified through extensive experiments on two few-shot benchmark MiniImageNet and CIFAR-FS dataset. The results suggest that our method has a strong capability of noise immunity and quick convergence. The classification accuracy outperforms most state-of-the-art approaches.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Domain Adaptation and Few-Shot Learning · Advanced Neural Network Applications · Machine Learning and ELM
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (45)
Show source-provided references
1. Adult image and video recognition by a deep multi context network and fine-to-coarse strategy10.1145/3057733 ↗
2. Deep supervised hashing based on stable distribution10.1109/ACCESS.2019.2900489 ↗
3. Balanced Deep Supervised Hashing10.32604/cmc.2019.05588 ↗
4. Robust mutual learning hashing
5. Person retrieval in surveillance videos via deep attribute mining and reasoning
6. Learning refined attribute-aligned network with attribute selection for person re-identification10.1016/j.neucom.2020.03.057 ↗
7. Improving person re-identification by multi-task learning10.1016/j.neucom.2019.01.027 ↗
8. Adaptive and robust partition learning for person retrieval with policy gradient
9. Deep learning face representation by fixed erasing in facial landmarks10.1007/s11042-019-07892-8 ↗
10. Learning from one example through shared densities on transforms
11. Few-shot learning via embedding adaptation with set-to-set functions
12. Prototypical networks for few-shot learning
13. Learning to compare: Relation network for few-shot learning
14. Dynamic few-shot visual learning without forgetting
15. Matching networks for one shot learning
16. Model-agnostic meta-learning for fast adaptation of deep networks
17. P. Battaglia, B. Hamrick, V. Bapst, A. Sanchez, V. Zambaldi et al., “Relational inductive biases, deep learning, and graph networks,” arXiv preprint, 2018.
18. Few-shot learning with graph neural networks
19. Edge-labeling graph neural network for few-shot learning
20. Learning to propagate labels: Transductive propagation network for few-shot learning
21. DPGN: Distribution propagation graph network for few-shot learning
22. Few-shot learning with global class representations
23. A. Nichol, J. Achiam and J. Schulman, “On first-order meta-learning algorithms,” arXiv preprint, 2018.
24. Task agnostic meta-learning for few-shot learning
25. Bayesian model-agnostic meta-learning
26. Optimization as a model for few-shot learning
27. Meta-learning probabilistic inference for prediction
28. Few-shot image recognition by predicting parameters from activations
29. Meta-learning with differentiable closed form solvers
30. Dynamic few-shot visual learning without forgetting
31. Attention is all you need
32. Gated graph sequence neural networks
33. Graph attention networks
34. Gaan: Gated attention networks for learning on large and spatiotemporal graphs
35. Few-shot learning with embedded class models and shot-free meta training
36. Meta-learning with differentiable convex optimization
37. Boosting few-shot visual learning with self-supervision
38. Z. Li, F. Zhou, F. Chen and H. Li, “Meta-SGD: Learning to learn quickly for few shot learning,” arXiv preprint, 2017.
39. Distribution consistency based covariance metric networks for few-shot learning
40. Meta learning with latent embedding optimization
41. A simple neural attentive meta-learner
42. H. Ye, H. Hu, D. Zhan and F. Sha, “Learning embedding adaptation for few-shot learning,” arXiv preprint, 2018.
43. A closer look at few-shot classification
44. Finding task-relevant features for few-shot learning by category traversal
45. Generating classification weights with gnn denoising autoencoders for few-shot learning