DDRCN: Deep Deterministic Policy Gradient Recommendation Framework Fused with Deep Cross Networks
Source abstract
Abstract
As an essential branch of artificial intelligence, recommendation systems have gradually penetrated people’s daily lives. It is the active recommendation of goods or services of potential interest to users based on their preferences. Many recommendation methods have been proposed in both industry and academia. However, there are some limitations of previous recommendation methods: (1) Most of them do not consider the cross-correlation between data. (2) Many treat the recommendation process as a one-time act and do not consider the continuity of the recommendation system. To overcome these limitations, we propose a recommendation framework based on deep reinforcement learning techniques, known as DDRCN: a deep deterministic policy gradient recommendation framework incorporating deep cross networks. We use a Deep network and a Cross network to fit the cross relationships between the data, to obtain a representation of the user interaction data. The Actor-Critic network is designed to simulate the continuous interaction behavior of users through a greedy strategy. A deep deterministic policy gradient network is also used to train the recommendation model. Finally, we conduct experiments with two publicly available datasets and find that our proposed recommendation framework outperforms the baseline approach in the recall and ranking phases of recommendations.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Recommender Systems and Techniques · Advanced Bandit Algorithms Research · Advanced Graph Neural Networks
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Software College, Northeastern University, Shenyang 110169, China
Science and Technology on Special System Simulation Laboratory, Beijing Simulation Center, Beijing 100854, China
Funding
National Natural Science Foundation of China · Fundamental Research Funds for the Central Universities
References (27)
Show source-provided references
1. A survey of multimedia recommendation systems: Challenges and opportunities10.1142/S1793351X17500039 ↗
2. The netflix recommendation system: Algorithms, business value, and innovation
3. Seide, F., Li, G., Chen, X., and Yu, D. (2011, January 11–15). Feature engineering in context-dependent deep neural networks for conversational speech transcription. Proceedings of the IEEE Workshop on Automatic Speech Recognition & Understanding, Waikoloa, HI, USA.10.1109/ASRU.2011.6163899 ↗
4. Wang, R., Fu, B., Fu, G., and Wang, M. (2017, January 14). Deep & cross network for ad click predictions. Proceedings of the ADKDD’17, Halifax, NS, Canada.10.1145/3124749.3124754 ↗
5. Wang, R., Shivanna, R., Cheng, D., Jain, S., Lin, D., Hong, L., and Chi, E. (2021, January 19–23). Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. Proceedings of the Web Conference, Ljubljana, Slovenia.10.1145/3442381.3450078 ↗
6. Xie, R., Wang, Y., Wang, R., Lu, Y., Zou, Y., Xia, F., and Lin, L. (2022, January 21–25). Long short-term temporal meta-learning in online recommendation. Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, Virtual Event, AZ, USA.10.1145/3488560.3498371 ↗
7. Shivaram, K., Liu, P., Shapiro, M., Bilgic, M., and Culotta, A. (2022, January 18–23). Reducing Cross-Topic Political Homogenization in Content-Based News Recommendation. Proceedings of the 16th ACM Conference on recommendation Systems, Seattle, WA, USA.10.1145/3523227.3546782 ↗
8. Rendle, S. (2010, January 13–17). Factorization machines. Proceedings of the 2010 IEEE International conference on data mining, Sydney, Australia.10.1109/ICDM.2010.127 ↗
9. Purushotham, S., Liu, Y., and Kuo, J. (July, January 26). Collaborative topic regression with social matrix factorization for recommendation systems. Proceedings of the International Conference on Machine Learning, Edinburgh, UK.
10. Montanés, E., Quevedo, J.R., Díaz, I., and Ranilla, J. (2009, January 7–11). Collaborative tag recommendation system based on logistic regression. Proceedings of the European Conference on Machine Learning/ Principles and Practice of Knowledge Discovery in Databases, Bled, Slovenia.
11. Sun, Y., Pan, J., Zhang, A., and Flores, A. (2021, January 19–23). FM2: Field-matrixed factorization machines for recommendation systems. Proceedings of the Web Conference 2021, Ljubljana, Slovenia.10.1145/3442381.3449930 ↗
12. Sedhain, S., Menon, A.K., Sanner, S., and Xie, L. (2015, January 18–22). Autorec: Autoencoders meet collaborative filtering. Proceedings of the 24th international conference on World Wide Web, Florence, Italy.10.1145/2740908.2742726 ↗
13. Cheng, H.T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., and Ispir, M. (2016, January 15). Wide & deep learning for recommendation systems. Proceedings of the 1st workshop on deep learning for recommendation systems, Boston, MA, USA.10.1145/2988450.2988454 ↗
14. Zhou, G., Mou, N., Fan, Y., Pi, Q., Bian, W., Zhou, C., Zhu, X., and Gai, K. Deep interest evolution network for click-through rate prediction. Proceedings of the AAAI Conference on Artificial Intelligence.
15. Liu, H., Wei, Y., Yin, J., and Nie, L. (2022). IEEE Transactions on Knowledge and Data Engineering, IEEE.
16. Recommendation by users’ multimodal preferences for smart city applications10.1109/TII.2020.3008923 ↗
17. Wang, K., Zou, Z., Deng, Q., Tao, J., Wu, R., Fan, C., Chen, L., and Cui, P. (2021). Reinforcement learning with a disentangled universal value function for item recommendation. InProceedings of the AAAI Conference on Artificial Intelligence, Association for the Advancement of Artificial Intelligence.10.1609/aaai.v35i5.16569 ↗
18. Zou, L., Xia, L., Ding, Z., Song, J., Liu, W., and Yin, D. (2019, January 4–8). Reinforcement learning to optimize long-term user engagement in recommendation systems. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.10.1145/3292500.3330668 ↗
19. Zou, L., Xia, L., Du, P., Zhang, Z., Bai, T., Liu, W., Nie, J.Y., and Yin, D. (2020, January 3–7). Pseudo Dyna-Q: A reinforcement learning framework for interactive recommendation. Proceedings of the 13th International Conference on Web Search and Data Mining, Houston, TX, USA.10.1145/3336191.3371801 ↗
20. Ma, J., Zhao, Z., Yi, X., Yang, J., Chen, M., Tang, J., Hong, L., and Chi, E.H. (2020, January 20–24). Off-policy learning in two-stage recommendation systems. Proceedings of The Web Conference 2020, Taipei, Taiwan.10.1145/3366423.3380130 ↗
21. Xin, X., Karatzoglou, A., Arapakis, I., and Jose, J.M. (2022, January 21–25). Supervised Advantage Actor-Critic for Recommender Systems. Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, Virtual Event, AZ, USA.10.1145/3488560.3498494 ↗
22. Liu, F., Tang, R., Li, X., Zhang, W., Ye, Y., Chen, H., Guo, H., and Zhang, Y. (2018). Deep reinforcement learning based recommendation with explicit user-item interactions modeling. arXiv.
23. Deterministic policy gradient algorithms
24. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2021). Continuous control with deep reinforcement learning. arXiv.
25. Li, Z., Zhao, H., Liu, Q., Huang, Z., Mei, T., and Chen, E. (2018, January 19–23). Learning from history and present: Next-item recommendation via discriminatively exploiting user behaviors. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.10.1145/3219819.3220014 ↗
26. Rendle, S., Freudenthaler, C., Gantner, Z., and Schmidt-Thieme, L. (2009, January 18–21). BPR: Bayesian personalized ranking from implicit feedback. Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, Montreal, QC, Canada.
27. He, X., Liao, L., Zhang, H., Nie, L., Hu, X., and Chua, T.S. (2017, January 3–7). Neural collaborative filtering. Proceedings of the 26th International Conference on World Wide Web, Perth, Australia.10.1145/3038912.3052569 ↗