journal article·2024-11-06Open access

UPGCN: User Perception-Guided Graph Convolutional Network for Multimodal Recommendation

Baihu Zhou, Yongquan Liang

Applied Sciences· Volume 14· Issue 22

Abstract

To tackle the challenges of cold start and data sparsity in recommendation systems, an increasing number of researchers are integrating item features, resulting in the emergence of multimodal recommendation systems. Although graph convolutional network-based approaches have achieved significant success, they still face two limitations: (1) Users have different preferences for various types of features, but existing methods often treat these preferences equally or fail to specifically address this issue. (2) They do not effectively distinguish the similarity between different modality item features, overlook the unique characteristics of each type, and fail to fully exploit their complementarity. To solve these issues, we propose the user perception-guided graph convolutional network for multimodal recommendation (UPGCN). This model consists of two main parts: the user perception-guided representation enhancement module (UPEM) and the multimodal two-step enhanced fusion method, which are designed to capture user preferences for different modalities to enhance user representation. At the same time, by distinguishing the similarity between different modalities, the model filters out noise and fully leverages their complementarity to achieve more accurate item representations. We performed comprehensive experiments on the proposed model, and the results indicate that it outperforms other baseline models in recommendation performance, strongly demonstrating its effectiveness.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Recommender Systems and Techniques · Topic Modeling · Image Retrieval and Classification Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao 266000, China

References (45)

Show source-provided references
1. Steffen, R., Christoph, F., Zeno, G., and Lars, S. (2009, January 18–21). BPR: Bayesian Personalized Ranking from Implicit Feedback. Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, Montreal, QC, Canada.
2. Wang, X., He, X., Wang, M., Feng, F., and Chua, T.S. (2019, January 21–25). Neural graph collaborative filtering. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France.10.1145/3331184.3331267 ↗
3. Wu, C., Wu, F., Qi, T., Zhang, C., Huang, Y., and Xu, T. (2022, January 11–15). MM-rec: Visiolinguistic model empowered multimodal news recommendation. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain.10.1145/3477495.3531896 ↗
4. He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., and Wang, M. (2020, January 25–30). Lightgcn: Simplifying and powering graph convolution network for recommendation. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event.10.1145/3397271.3401063 ↗
5. He, R., and McAuley, J. (2016, January 12–17). VBPR: Visual bayesian personalized ranking from implicit feedback. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.9973 ↗
6. Sun, R., Cao, X., Zhao, Y., Wan, J., Zhou, K., Zhang, F., and Zheng, K. (2020, January 19–23). Multi-modal knowledge graphs for recommender systems. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, New York, NY, USA.10.1145/3340531.3411947 ↗
7. MEGCF: Multimodal entity graph collaborative filtering for personalized recommendation10.1145/3544106 ↗
8. Hierarchical user intent graph network for multimedia recommendation10.1109/TMM.2021.3088307 ↗
9. Heterogeneous graph contrastive learning network for personalized micro-video recommendation10.1109/TMM.2022.3151026 ↗
10. Mu, Z., Zhuang, Y., Tan, J., Xiao, J., and Tang, S. (2022, January 10–14). Learning hybrid behavior patterns for multimedia recommendation. Proceedings of the 30th ACM International Conference on Multimedia, Seattle, WA, USA.10.1145/3503161.3548119 ↗
11. Yi, Z., Wang, X., Ounis, I., and Macdonald, C. (2022, January 11–15). Multi-modal graph contrastive learning for micro-video recommendation. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain.10.1145/3477495.3532027 ↗
12. Ye, X., Cai, G., and Song, Y. (2022, January 23–25). Multi-modal Personalized Goods Recommendation based on Graph Enhanced Attention GNN. Proceedings of the 2022 5th International Conference on Machine Learning and Machine Intelligence, Hangzhou, China.10.1145/3568199.3568223 ↗
13. Wei, Y., Wang, X., Nie, L., He, X., Hong, R., and Chua, T.S. (2019, January 21–25). MMGCN: Multi-modal graph convolution network for personalized recommendation of micro-video. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.10.1145/3343031.3351034 ↗
14. Wei, Y., Wang, X., Nie, L., He, X., and Chua, T.S. (2020, January 12). Graph-refined convolutional network for multimedia recommendation with implicit feedback. Proceedings of the 28th ACM international Conference on Multimedia, Seattle, WA, USA.10.1145/3394171.3413556 ↗
15. Chen, F., Wang, J., Wei, Y., Zheng, H.T., and Shao, J. (2022, January 10–14). Breaking isolation: Multimodal graph fusion for multimedia recommendation by edge-wise modulation. Proceedings of the 30th ACM International Conference on Multimedia, Lisboa, Portugal.10.1145/3503161.3548399 ↗
16. Zhou, H., Zhou, X., Zhang, L., and Shen, Z. (2023). Enhancing Dyadic Relations with Homogeneous Graphs for Multimodal Recommendation. arXiv.10.3233/FAIA230631 ↗
17. Kim, T., Lee, Y.C., Shin, K., and Kim, S.W. (2022, January 17–21). MARIO: Modality-aware attention and modality-preserving decoders for multimedia recommendation. Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA.10.1145/3511808.3557387 ↗
18. Cui, X., Qu, X., Li, D., Yang, Y., Li, Y., and Zhang, X. (2023). MKGCN: Multi-modal knowledge graph convolutional network for music recommender systems. Electronics, 12.10.3390/electronics12122688 ↗
19. Liu, Y., Yang, S., Lei, C., Wang, G., Tang, H., Zhang, J., Sun, A., and Miao, C. (2021, January 20–24). Pre-training graph transformer with multimodal side information for recommendation. Proceedings of the 29th ACM International Conference on Multimedia, Virtual Event.10.1145/3474085.3475709 ↗
20. Trust-aware multi-task knowledge graph for recommendation
21. Disentangled multimodal representation learning for recommendation10.1109/TMM.2022.3217449 ↗
22. Learning the user’s deeper preferences for multi-modal recommendation systems10.1145/3573010 ↗
23. DualGNN: Dual graph neural network for multimedia recommendation10.1109/TMM.2021.3138298 ↗
24. Zhang, J., Zhu, Y., Liu, Q., Wu, S., Wang, S., and Wang, L. (2021, January 20–24). Mining latent structures for multimedia recommendation. Proceedings of the 29th ACM International Conference on Multimedia, Virtual Event.10.1145/3474085.3475259 ↗
25. Zhou, X., and Shen, Z. (2020, January 12–16). A tale of two graphs: Freezing and denoising graph structures for multimodal recommendation. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada.
26. MGAT: Multimodal graph attention network for recommendation10.1016/j.ipm.2020.102277 ↗
27. Zheng, L., Noroozi, V., and Yu, P.S. (2017, January 6–10). Joint deep modeling of users and items using reviews for recommendation. Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, Cambridge, UK.10.1145/3018661.3018665 ↗
28. Kang, W.C., Fang, C., Wang, Z., and McAuley, J. (2017, January 18–21). Visually-aware fashion recommendation and design with generative image models. Proceedings of the 2017 IEEE International Conference on Data Mining (ICDM), New Orleans, LA, USA.10.1109/ICDM.2017.30 ↗
29. Multimodal representation learning for recommendation in Internet of Things10.1109/JIOT.2019.2940709 ↗
30. Chen, X., Chen, H., Xu, H., Zhang, Y., Cao, Y., Qin, Z., and Zha, H. (2019, January 21–25). Personalized fashion recommendation with visual explanations based on multimodal attention network: Towards visually explainable recommendation. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France.10.1145/3331184.3331254 ↗
31. Chen, J., Zhang, H., He, X., Nie, L., Liu, W., and Chua, T.S. (2017, January 7–11). Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention. Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, Tokyo, Japan.10.1145/3077136.3080797 ↗
32. Kipf, T.N., and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv.
33. Syntax-guided text generation via graph neural network10.1007/s11432-019-2740-1 ↗
34. M2GCN: Multi-modal graph convolutional network for modeling polypharmacy side effects10.1007/s10489-022-03839-z ↗
35. Latent structure mining with contrastive modality fusion for multimedia recommendation10.1109/TKDE.2022.3221949 ↗
36. Zhou, X., Zhou, H., Liu, Y., Zeng, Z., Miao, C., Wang, P., and Jiang, F. (May, January 30). Bootstrap latent representations for multi-modal recommendation. Proceedings of the ACM Web Conference 2023, Austin, TX, USA.10.1145/3543507.3583251 ↗
37. Multimodal machine learning: A survey and taxonomy
38. Liu, S., Chen, Z., Liu, H., and Hu, X. (2019, January 13–17). User-video co-attention network for personalized micro-video recommendation. Proceedings of the World Wide Web Conference, San Francisco, CA, USA.10.1145/3308558.3313513 ↗
39. Self-supervised learning for multimedia recommendation10.1109/TMM.2022.3187556 ↗
40. Wang, Y., Xu, X., Yu, W., Xu, R., Cao, Z., and Shen, H.T. (2021, January 10–12). Combine early and late fusion together: A hybrid fusion framework for image-text matching. Proceedings of the 2021 IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, China.10.1109/ICME51207.2021.9428201 ↗
41. Fast Approximate kNN Graph Construction for High Dimensional Data via Recursive Lanczos Bisection
42. McAuley, J., Targett, C., Shi, Q., and Van Den Hengel, A. (2015, January 9–13). Image-based recommendations on styles and substitutes. Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile.10.1145/2766462.2767755 ↗
43. Glorot, X., and Bengio, Y. (2010, January 13–15). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Chia Laguna Resort, Sardinia, Italy.
44. Kingma, D.P. (2014). Adam: A method for stochastic optimization. arXiv.
45. Learning an evolutionary embedding via massive knowledge distillation10.1007/s11263-019-01286-x ↗