Contextual Feature Expansion with Superordinate Concept for Compositional Zero-Shot Learning
Source abstract
Abstract
Compositional Zero-Shot Learning (CZSL) seeks to enable machines to recognize objects and attributes (i.e., primitives),learn their associations, and generalize to novel compositions, enabling systems to exhibit a human-like ability to infer and generalize. The existing approaches, multi-label and multi-class classification, face inherent trade-offs: the former suffers from biases against unrelated compositions, while the latter struggles with exponentially growing search spaces as the number of objects and attributes increases. To overcome these limitations and address the exponential complexity in CZSL, we introduce Concept-oriented Feature ADjustment (CoFAD), a novel method that extracts superordinate conceptual features based on primitive relationships and expands label feature boundaries. By incorporating spectral clustering and membership function in fuzzy logic, CoFAD achieves state-of-the-art performance while using 2×–4× less GPU memory and reducing training time by up to 50× on large-scale dataset.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Domain Adaptation and Few-Shot Learning · COVID-19 diagnosis using AI
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Artificial Intelligence, Hanyang University, Seoul 04763, Republic of Korea
Funding
Institute of Information and communications Technology Planning and evaluation (IITP) · National Research Foundation of Korea (NRF)
References (33)
Show source-provided references
1. Basic objects in natural categories10.1016/0010-0285(76)90013-X ↗
2. Rosch, E. (1978). Principles of categorization. Cognition and Categorization/Erlbaum, Routledge.
3. Misra, I., Gupta, A., and Hebert, M. (2017, January 21–26). From red wine to red tomato: Composition with context. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.129 ↗
4. Nagarajan, T., and Grauman, K. (2018, January 8–14). Attributes as operators: Factorizing unseen attribute-object compositions. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01246-5_11 ↗
5. Purushwalkam, S., Nickel, M., Gupta, A., and Ranzato, M. (November, January 27). Task-driven modular networks for zero-shot compositional learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Rpublic of Korea.
6. Li, Y.L., Xu, Y., Mao, X., and Lu, C. (2020, January 13–19). Symmetry and group in attribute-object compositions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01133 ↗
7. Naeem, M.F., Xian, Y., Tombari, F., and Akata, Z. (2021, January 20–25). Learning graph embeddings for compositional zero-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00101 ↗
8. Learning graph embeddings for open world compositional zero-shot learning10.1109/TPAMI.2022.3163667 ↗
9. Karthik, S., Mancini, M., and Akata, Z. (2021, January 6–14). Revisiting visual product for compositional zero-shot learning. Proceedings of the NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications, Virtual.10.1109/CVPR46437.2021.00518 ↗
10. Focusing on valid search space in Open-World Compositional Zero-Shot Learning by leveraging misleading answers10.1109/ACCESS.2024.3491174 ↗
11. Anwaar, M.U., Pan, Z., and Kleinsteuber, M. (2022, January 10–14). On leveraging variational graph embeddings for open world compositional zero-shot learning. Proceedings of the 30th ACM International Conference on Multimedia, Lisbon, Portugal.10.1145/3503161.3547798 ↗
12. Karthik, S., Mancini, M., and Akata, Z. (2022, January 18–24). Kg-sp: Knowledge guided simple primitives for open world compositional zero-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00912 ↗
13. Li, X., Yang, X., Wei, K., Deng, C., and Yang, M. (2022, January 18–24). Siamese contrastive embedding network for compositional zero-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00911 ↗
14. Huang, S., Gong, B., Feng, Y., Zhang, M., Lv, Y., and Wang, D. (2024, January 16–22). Troika: Multi-path cross-modal traction for compositional zero-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.02266 ↗
15. Li, Y., Liu, Z., Chen, H., and Yao, L. (2024, January 16–22). Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.01612 ↗
16. Isola, P., Lim, J.J., and Adelson, E.H. (2015, January 7–12). Discovering states and transformations in image collections. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298744 ↗
17. Yu, A., and Grauman, K. (2014, January 23–28). Fine-grained visual comparisons with local learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.32 ↗
18. Hao, S., Han, K., and Wong, K.Y.K. (2023, January 17–24). Learning attention as disentangler for compositional zero-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01470 ↗
19. Mancini, M., Naeem, M.F., Xian, Y., and Akata, Z. (2021, January 19–25). Open world compositional zero-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00518 ↗
20. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18–24). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning PMLR, Online.
21. Nayak, N.V., Yu, P., and Bach, S.H. (2022). Learning to compose soft prompts for compositional zero-shot learning. arXiv.
22. Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. (2019, January 9–15). Parameter-efficient transfer learning for NLP. Proceedings of the International Conference on Machine Learning PMLR, Long Beach, CA, USA.
23. A tutorial on spectral clustering10.1007/s11222-007-9033-z ↗
24. Hudson, D.A., and Manning, C.D. (2019, January 15–20). Gqa: A new dataset for real-world visual reasoning and compositional question answering. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00686 ↗
25. Zhang, T., Liang, K., Du, R., Sun, X., Ma, Z., and Guo, J. (2022). Learning invariant visual representations for compositional zero-shot learning. Proceedings of the European Conference on Computer Vision, Springer.10.1007/978-3-031-20053-3_20 ↗
26. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). Pytorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst., 32.
27. Learning to prompt for vision-language models10.1007/s11263-022-01653-1 ↗
28. Xu, G., Kordjamshidi, P., and Chai, J. (2022). Prompting large pre-trained vision-language models for compositional concept learning. arXiv.
29. Wang, H., Yang, M., Wei, K., and Deng, C. (2023, January 19–25). Hierarchical prompt learning for compositional zero-shot recognition. Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, Macao, China.10.24963/ijcai.2023/163 ↗
30. Xu, G., Chai, J., and Kordjamshidi, P. (2024, January 3–8). GIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learning. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV57701.2024.00567 ↗
31. Lu, X., Guo, S., Liu, Z., and Guo, J. (2023, January 23–24). Decomposed soft prompt guided fusion enhancing for compositional zero-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.02256 ↗
32. Bao, W., Chen, L., Huang, H., and Kong, Y. (October, January 29). Prompting language-informed distribution for compositional zero-shot learning. Proceedings of the 18th European Conference on Computer Vision, Milan, Italy.
33. Saini, N., Pham, K., and Shrivastava, A. (2022, January 18–24). Disentangling visual embeddings for attributes and objects. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01329 ↗