GC4MRec: Generative-Contrastive for Multimodal Recommendation
Source abstract
Abstract
The rapid growth of information technology has led to an explosion of data, posing a significant challenge for data processing. Recommendation systems aim to address this by providing personalized content recommendations to users from vast datasets. Recently, multimodal recommendation systems have gained considerable attention due to their ability to leverage diverse data modalities (e.g., images and text) for more accurate recommendations. However, effectively fusing these modalities to accurately represent user preferences remains a challenging task, despite progress made by existing multimodal recommendation approaches. To address this challenge, we propose a novel method which we call GC4MRec (Generative-Contrastive for Multimodal Recommendation). On the one hand, we design a bilateral information flow module using two graph convolutional networks (GCNs). This module captures modal features from two distinct perspectives—standard and generatively augmented—to extract latent preferences. On the other hand, we introduce a novel modality fusion module that dynamically represents user multimodal fusion preferences, enabling the construction of accurate user preference profiles. Finally, we evaluate our proposed method, GC4MRec, on three public real-world datasets and demonstrate its effectiveness compared to the state-of-the-art methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Recommender Systems and Techniques · Topic Modeling · Image Retrieval and Classification Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Management Science and Information Engineering, Jilin University of Finance and Economics, Changchun 130117, China
Funding
National Natural Science Foundation of China
References (46)
Show source-provided references
1. Lops, P., de Gemmis, M., and Semeraro, G. (2011). Content-based Recommender Systems: State of the Art and Trends. Recommender Systems Handbook, Spring.10.1007/978-0-387-85820-3_3 ↗
2. Mooney, R.J., and Roy, L. (2000, January 2–7). Content-based book recommending using learning for text categorization. Proceedings of the Fifth ACM Conference on Digital Libraries, ACM, San Antonio, TX, USA. DL00.10.1145/336597.336662 ↗
3. Koren, Y., Rendle, S., and Bell, R. (2021). Advances in collaborative filtering. Recommender Systems Handbook, Spring.10.1007/978-1-0716-2197-4_3 ↗
4. Using collaborative filtering to weave an information tapestry10.1145/138859.138867 ↗
5. Breese, J.S., Heckerman, D., and Kadie, C. (2013). Empirical analysis of predictive algorithms for collaborative filtering. arXiv.
6. Sarwar, B., Karypis, G., Konstan, J., and Riedl, J. (2001, January 1–5). Item-based collaborative filtering recommendation algorithms. Proceedings of the 10th international conference on World Wide Web, Hong Kong, China.10.1145/371920.372071 ↗
7. Kipf, T.N., and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv.
8. He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., and Wang, M. (2020, January 25–30). Lightgcn: Simplifying and powering graph convolution network for recommendation. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Xi’an, China.10.1145/3397271.3401063 ↗
9. Liu, F., Cheng, Z., Zhu, L., Gao, Z., and Nie, L. (2021, January 12–16). Interest-aware message-passing GCN for recommendation. Proceedings of the Web Conference 2021, Ljubljana, Slovenia.10.1145/3442381.3449986 ↗
10. Peng, S., Sugiyama, K., and Mine, T. (2022, January 17–21). SVD-GCN: A simplified graph convolution paradigm for recommendation. Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA.10.1145/3511808.3557462 ↗
11. Wu, L., Sun, P., Hong, R., Fu, Y., Wang, X., and Wang, M. (2018). Socialgcn: An efficient graph convolutional network based model for social recommendation. arXiv.
12. Wei, Y., Wang, X., Nie, L., He, X., Hong, R., and Chua, T.S. (2019, January 21–25). MMGCN: Multi-modal graph convolution network for personalized recommendation of micro-video. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.10.1145/3343031.3351034 ↗
13. Zhang, J., Zhu, Y., Liu, Q., Wu, S., Wang, S., and Wang, L. (2021, January 20–24). Mining latent structures for multimedia recommendation. Proceedings of the 29th ACM International Conference on Multimedia, Online.10.1145/3474085.3475259 ↗
14. Disentangled graph variational auto-encoder for multimodal recommendation with interpretability10.1109/TMM.2024.3369875 ↗
15. Zhou, H., Zhou, X., Zhang, L., and Shen, Z. (2023). Enhancing dyadic relations with homogeneous graphs for multimodal recommendation. ECAI 2023, IOS Press.10.3233/FAIA230631 ↗
16. Multimodal graph contrastive learning for multimedia-based recommendation10.1109/TMM.2023.3251108 ↗
17. Zhou, B., and Liang, Y. (2024). UPGCN: User Perception-Guided Graph Convolutional Network for Multimodal Recommendation. Appl. Sci., 14.10.3390/app142210187 ↗
18. Cui, X., Qu, X., Li, D., Yang, Y., Li, Y., and Zhang, X. (2023). Mkgcn: Multi-modal knowledge graph convolutional network for music recommender systems. Electronics, 12.10.3390/electronics12122688 ↗
19. Zhou, X., and Shen, Z. (3, January 29). A tale of two graphs: Freezing and denoising graph structures for multimodal recommendation. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada.
20. Time-frequency sensitive prompt tuning framework for session-based recommendation10.1016/j.eswa.2025.126501 ↗
21. User identification network with contrastive clustering for shared-account recommendation10.1016/j.ipm.2024.104055 ↗
22. Zhou, C., Zhou, S., Huang, J., and Wang, D. (2024). Hierarchical Self-Supervised Learning for Knowledge-Aware Recommendation. Appl. Sci., 14.10.3390/app14209394 ↗
23. Ma, J., Wan, Y., and Ma, Z. (2024). Memory-Based Learning and Fusion Attention for Few-Shot Food Image Generation Method. Appl. Sci., 14.10.3390/app14188347 ↗
24. Kipf, T.N., and Welling, M. (2016). Variational graph auto-encoders. arXiv.
25. Hu, J., Shen, L., and Sun, G. (2018, January 18–22). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
26. He, R., and McAuley, J. (2016, January 12–17). VBPR: Visual bayesian personalized ranking from implicit feedback. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.9973 ↗
27. Liu, F., Cheng, Z., Sun, C., Wang, Y., Nie, L., and Kankanhalli, M. (2019, January 21–25). User diverse preference modeling by multimodal attentive metric learning. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.10.1145/3343031.3350953 ↗
28. Chen, J., Zhang, H., He, X., Nie, L., Liu, W., and Chua, T.S. (2017, January 7–11). Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention. Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan.10.1145/3077136.3080797 ↗
29. Mgat: Multimodal graph attention network for recommendation10.1016/j.ipm.2020.102277 ↗
30. Wei, W., Huang, C., Xia, L., and Zhang, C. (4, January 30). Multi-modal self-supervised learning for recommendation. Proceedings of the ACM Web Conference 2023, Austin, TX, USA.10.1145/3543507.3583206 ↗
31. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18–24). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning, PMLR, Virtual. Available online: https://proceedings.mlr.press/v139/radford21a.
32. Wu, J., Wang, X., Feng, F., He, X., Chen, L., Lian, J., and Xie, X. (2021, January 11–15). Self-supervised graph learning for recommendation. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event.10.1145/3404835.3462862 ↗
33. Xie, X., Sun, F., Liu, Z., Wu, S., Gao, J., Zhang, J., Ding, B., and Cui, B. (2022, January 9–12). Contrastive learning for sequential recommendation. Proceedings of the 2022 IEEE 38th International Conference on Data Engineering (ICDE), IEEE, Virtual Event.10.1109/ICDE53745.2022.00099 ↗
34. Xia, X., Yin, H., Yu, J., Wang, Q., Cui, L., and Zhang, X. (2021, January 2–9). Self-supervised hypergraph convolutional networks for session-based recommendation. Proceedings of the AAAI conference on artificial intelligence, Virtual Event.10.1609/aaai.v35i5.16578 ↗
35. Yang, Y., Wu, L., Hong, R., Zhang, K., and Wang, M. (2021, January 11–15). Enhanced graph learning for collaborative filtering via mutual information maximization. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event.10.1145/3404835.3462928 ↗
36. Lin, Z., Tian, C., Hou, Y., and Zhao, W.X. (2022, January 25–29). Improving graph collaborative filtering with neighborhood-enriched contrastive learning. Proceedings of the ACM Web Conference 2022, Virtual Event.10.1145/3485447.3512104 ↗
37. Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion
38. Xie, Z., Liu, C., Zhang, Y., Lu, H., Wang, D., and Ding, Y. (2021, January 19–23). Adversarial and contrastive variational autoencoder for sequential recommendation. Proceedings of the Web Conference 2021, Ljubljana, Slovenia.10.1145/3442381.3449873 ↗
39. Zhu, Y., Wu, L., Guo, Q., Hong, L., and Li, J. (2024, January 13–17). Collaborative large language model for recommender systems. Proceedings of the ACM on Web Conference 2024, Singapore.10.1145/3589334.3645347 ↗
40. Sun, F., Liu, J., Wu, J., Pei, C., Lin, X., Ou, W., and Jiang, P. (2019, January 3–7). BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China.10.1145/3357384.3357895 ↗
41. Hao, Y., Zhao, P., Fang, J., Qu, J., Liu, G., Zhuang, F., Sheng, V.S., and Zhou, X. (2024, January 13–17). Meta-optimized joint generative and contrastive learning for sequential recommendation. Proceedings of the 2024 IEEE 40th International Conference on Data Engineering (ICDE), IEEE, Utrecht, The Netherlands.10.1109/ICDE60146.2024.00060 ↗
42. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., and Hu, Q. (2020, January 14–19). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01155 ↗
43. Rendle, S., Freudenthaler, C., Gantner, Z., and Schmidt-Thieme, L. (2012). BPR: Bayesian personalized ranking from implicit feedback. arXiv.
44. Self-supervised learning for multimedia recommendation10.1109/TMM.2022.3187556 ↗
45. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). Pytorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst., 32, Available online: https://proceedings.neurips.cc/paper/2019/hash/bdbca288fee7f92f2bfa9f7012727740-Abstract.html.
46. Kingma, D.P. (2014). Adam: A method for stochastic optimization. arXiv.