Deep Collaborative Recommendation Algorithm Based on Attention Mechanism
Source abstract
Abstract
Representation learning-based collaborative filtering (CF) methods address the linear relationship of user-items with dot products and cannot study the latent nonlinear relationship applied to implicit feedback. Matching function learning-based CF methods directly learn the complicated mapping functions that map user-item pairs to matching scores, which has limitations in identifying low-rank relationships. To this end, we propose a deep collaborative recommendation algorithm based on attention mechanism (DACR). First, before the user-item representations are input into the DNNs, we utilize the attention mechanism to adaptively assign different weights to the user-item representations, which captures the hidden information in implicit feedback. After that, we input the user-item representations with corresponding weights into the representation learning and matching function learning modules. Finally, we concatenate the prediction vectors learned from different dimensions to predict the matching scores. The results show that we can improve the expression ability of the model while taking into account not only the nonlinear information hidden in implicit feedback, but also the low-rank relationships of user-item pairs to obtain more accurate predictions. Through detailed experiments on two datasets, we find that the ranking capability of the DACR model is enhanced compared with other baseline models, and the evaluation metrics HR and NDCG of DACR are increased by 0.88–1.19% and 0.65–1.15%, respectively.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Recommender Systems and Techniques · Digital Marketing and Social Media · Advanced Technologies in Various Fields
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Science and Engineering, Xinjiang University, Urumqi 830046, China
Key Laboratory of Signal Detection and Processing, Xinjiang Uygur Autonomous Region, Xinjiang University, Urumqi 830046, China
Funding
National Science Foundation of China · Science Fund for Outstanding Youth of Xinjiang Uygur Autonomous Region · Key Laboratory Open Project of Science and Technology Department of Xinjiang Uygur Autonomous Region named Research on video information intelligent processing technology for Xinjiang regional security · Major science and technology project of Xinjiang Uygur Autonomous Region
References (47)
Show source-provided references
1. Zhao, Z.L., Huang, L., Wang, C.D., and Huang, D. (2018, January 21–24). Low-rank and sparse cross-domain recommendation algorithm. Proceedings of the International Conference on Database Systems for Advanced Applications, Gold Coast, QLD, Australia.10.1007/978-3-319-91452-7_10 ↗
2. Typicality-based collaborative filtering recommendation10.1109/TKDE.2013.7 ↗
3. Relational collaborative topic regression for recommender systems10.1109/TKDE.2014.2365789 ↗
4. Item orientated recommendation by multi-view intact space learning with overlapping10.1016/j.knosys.2018.11.005 ↗
5. Applications of ML/DL in the management of smart cities and societies based on new trends in information technologies: A systematic literature review10.1016/j.scs.2022.104089 ↗
6. Heidi, A., Jafari Navimipour, N., Unal, M., and Toumaj, S. (2022). Machine learning applications for COVID-19 outbreak management. Neural Comput. Appl., 1–36.10.1007/s00521-022-07424-w ↗
7. Covington, P., Adams, J., and Sargin, E. (2016, January 15–19). Deep neural networks for youtube recommendations. Proceedings of the 10th ACM Conference on Recommender Systems, Boston, MA, USA.10.1145/2959100.2959190 ↗
8. Collaborative filtering recommendation algorithm based on attention GRU and adversarial learning10.1109/ACCESS.2020.3038770 ↗
9. Huang, P.S., He, X., Gao, J., Deng, L., Acero, A., and Heck, L. (November, January 27). Learning deep structured semantic models for web search using clickthrough data. Proceedings of the 22nd ACM International Conference on INFORMATION & Knowledge Management, San Francisco, CA, USA.
10. Xue, H.J., Dai, X., Zhang, J., Huang, S., and Chen, J. (2017, January 19–25). Deep matrix factorization models for recommender systems. Proceedings of the IJCAI, Melbourne, Australia.10.24963/ijcai.2017/447 ↗
11. Deep learning for matching in search and recommendation10.1561/1500000076 ↗
12. Multilayer feedforward networks are universal approximators10.1016/0893-6080(89)90020-8 ↗
13. He, X., Liao, L., Zhang, H., Nie, L., Hu, X., and Chua, T.S. (2017, January 3–7). Neural collaborative filtering. Proceedings of the 26th International Conference on World Wide Web, Perth, Australia.10.1145/3038912.3052569 ↗
14. Beutel, A., Covington, P., Jain, S., Xu, C., Li, J., Gatto, V., and Chi, E.H. (2018, January 5–9). Latent cross: Making use of context in recurrent recommender systems. Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, Los Angeles, CA, USA.10.1145/3159652.3159727 ↗
15. A review on the attention mechanism of deep learning10.1016/j.neucom.2021.03.091 ↗
16. Feng, J., Feng, X., Chen, J., Cao, X., Zhang, X., Jiao, L., and Yu, T. (2020). Generative adversarial networks based on collaborative learning and attention mechanism for hyperspectral image classification. Remote Sens., 12.10.3390/rs12071149 ↗
17. Wang, H., Zhang, F., Xie, X., and Guo, M. (2018, January 23–27). DKN: Deep knowledge-aware network for news recommendation. Proceedings of the 2018 World Wide Web Conference, Lyon, France.10.1145/3178876.3186175 ↗
18. Nais: Neural attentive item similarity model for recommendation10.1109/TKDE.2018.2831682 ↗
19. Tay, Y., Zhang, S., Tuan, L.A., and Hui, S.C. (2018). Self-attentive neural collaborative filtering. arXiv.
20. Xiao, J., Ye, H., He, X., Zhang, H., Wu, F., and Chua, T.S. (2017). Attentional factorization machines: Learning the weight of feature interactions via attention networks. arXiv.10.24963/ijcai.2017/435 ↗
21. Oard, D.W., and Kim, J. (1998, January 26–30). Implicit feedback for recommender systems. Proceedings of the AAAI Workshop on Recommender Systems, Madison, WI, USA. AAAI.
22. Ma, H. (August, January 28). An experimental study on implicit social recommendation. Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, Dublin, Ireland.
23. Hu, Y., Koren, Y., and Volinsky, C. (2008, January 15–19). Collaborative filtering for implicit feedback datasets. Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, Washington, DC, USA.10.1109/ICDM.2008.22 ↗
24. Koren, Y. (2008, January 24–27). Factorization meets the neighborhood: A multifaceted collaborative filtering model. Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, NV, USA.10.1145/1401890.1401944 ↗
25. Rendle, S., Freudenthaler, C., Gantner, Z., and Schmidt-Thieme, L. (2012). BPR: Bayesian personalized ranking from implicit feedback. arXiv.
26. Matrix factorization techniques for recommender systems10.1109/MC.2009.263 ↗
27. Koren, Y. (July, January 28). Collaborative filtering with temporal dynamics. Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France.
28. Hu, L., Sun, A., and Liu, Y. (2014, January 6–11). Your neighbors affect your ratings: On geographical neighborhood influence to rating prediction. Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, Gold Coast, QLD, Australia.10.1145/2600428.2609593 ↗
29. Wang, H., Wang, N., and Yeung, D.Y. (2015, January 10–13). Collaborative deep learning for recommender systems. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, Australia.10.1145/2783258.2783273 ↗
30. Sedhain, S., Menon, A.K., Sanner, S., and Xie, L. (2015, January 18–22). Autorec: Autoencoders meet collaborative filtering. Proceedings of the 24th International Conference on World Wide Web, Florence, Italy.10.1145/2740908.2742726 ↗
31. A deep learning based trust-and tag-aware recommender system10.1016/j.neucom.2021.11.064 ↗
32. Ahmadian, M., Ahmadi, M., Ahmadian, S., Jalali, S.M.J., Khosravi, A., and Nahavandi, S. (2021, January 17–20). Integration of Deep Sparse Autoencoder and Particle Swarm Optimization to Develop a Recommender System. Proceedings of the 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Melbourne, Australia.10.1109/SMC52423.2021.9658926 ↗
33. A reliable deep representation learning to improve trust-aware recommendation systems10.1016/j.eswa.2022.116697 ↗
34. Bai, T., Wen, J.R., Zhang, J., and Zhao, W.X. (2017, January 6–10). A neural collaborative filtering model with interaction-based neighborhood. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore.10.1145/3132847.3133083 ↗
35. Cheng, H.T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., and Ispir, M. (2016, January 15). Wide & deep learning for recommender systems. Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, Boston, MA, USA.10.1145/2988450.2988454 ↗
36. He, X., Du, X., Wang, X., Tian, F., Tang, J., and Chua, T.S. (2018). Outer product-based neural collaborative filtering. arXiv.10.24963/ijcai.2018/308 ↗
37. Zhang, Q., Cao, L., Zhu, C., Li, Z., and Sun, J. (2018, January 13–19). Coupledcf: Learning explicit and implicit user-item couplings in recommendation for deep collaborative filtering. Proceedings of the IJCAI International Joint Conference on Artificial Intelligence, Stockholm, Sweden.10.24963/ijcai.2018/509 ↗
38. Chen, J., Zhang, H., He, X., Nie, L., Liu, W., and Chua, T.S. (2017, January 7–11). Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Tokyo, Japan.10.1145/3077136.3080797 ↗
39. Cheng, Z., Ding, Y., He, X., Zhu, L., Song, X., and Kankanhalli, M.S. (2018, January 13–19). A3NCF: An Adaptive Aspect Attention Model for Rating Prediction. Proceedings of the IJCAI 2018, Stockholm, Sweden.10.24963/ijcai.2018/521 ↗
40. An adaptive deep learning method for item recommendation system10.1016/j.knosys.2020.106681 ↗
41. Xi, W.D., Huang, L., Wang, C.D., Zheng, Y.Y., and Lai, J. (2019, January 10–16). BPAM: Recommendation Based on BP Neural Network with Attention Mechanism. Proceedings of the IJCAI, Macao.10.24963/ijcai.2019/542 ↗
42. Pan, R., Zhou, Y., Cao, B., Liu, N.N., Lukose, R., Scholz, M., and Yang, Q. (2008, January 15–19). One-class collaborative filtering. Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, Washington, DC, USA.10.1109/ICDM.2008.16 ↗
43. Salakhutdinov, R., and Mnih, A. (2008, January 5–9). Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland.10.1145/1390156.1390267 ↗
44. McCaffrey, J.D. (2018). Why you should use cross-entropy error instead of classification error or mean squared error for neural network classifier training. Last Accessed Jan.
45. Sarwar, B., Karypis, G., Konstan, J., and Riedl, J. (2001, January 1–5). Item-based collaborative filtering recommendation algorithms. Proceedings of the 10th International Conference on World Wide Web, Hong Kong.10.1145/371920.372071 ↗
46. Deng, Z.H., Huang, L., Wang, C.D., Lai, J.H., and Philip, S.Y. (February, January 27). Deepcf: A unified framework of representation learning and matching function learning in recommender system. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
47. Deep attention user-based collaborative filtering for recommendation10.1016/j.neucom.2019.09.050 ↗