journal article·2024-01-30Open access

Integration of Deep Reinforcement Learning with Collaborative Filtering for Movie Recommendation Systems

Sony Peng, Sophort Siet, Sadriddinov Ilkhomjon, Dae-Young Kim, Doo-Soon Park

Applied Sciences· Volume 14· Issue 3

Abstract

In the era of big data, effective recommendation systems are essential for providing users with personalized content and reducing search time on online platforms. Traditional collaborative filtering (CF) methods face challenges like data sparsity and the new-user or cold-start issue, primarily due to their reliance on limited user–item interactions. This paper proposes an innovative movie recommendation system that integrates deep reinforcement learning (DRL) with CF, employing the actor–critic method and the Deep Deterministic Policy Gradient (DDPG) algorithm. This integration enhances the system’s ability to navigate the recommendation space effectively, especially for new users with less interaction data. The system utilizes DRL for making initial recommendations to new users and to generate optimal recommendation as more data becomes available. Additionally, singular value decomposition (SVD) is used for matrix factorization in CF, improving the extraction of detailed embeddings that capture the latent features of users and movies. This approach significantly increases recommendation precision and personalization. Our model’s performance is evaluated using the MovieLens dataset with metrics like Precision, Recall, and F1 Score and demonstrates its effectiveness compared with existing recommendation benchmarks, particularly in addressing sparsity and new-user challenges. Several benchmarks of existing recommendation models are selected for the purpose of model comparison.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Recommender Systems and Techniques · Advanced Bandit Algorithms Research

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Software Convergence, Soonchunhyang University, Asan-si 31538, Republic of Korea

Department of Computer Science Engineering, Soonchunhyang University, Asan-si 31538, Republic of Korea

Funding

National Research Foundation of Korea · Fostering Outstanding Universities

References (33)

Show source-provided references
1. Online serendipity: The case for curated recommender systems10.1016/j.bushor.2017.05.005 ↗
2. Survey on collaborative filtering, content-based filtering and hybrid recommendation system
3. Ferreira, D., Silva, S., Abelha, A., and Machado, J. (2020). Recommendation system using autoencoders. Appl. Sci., 10.10.3390/app10165510 ↗
4. A review on reinforcement learning for contact-rich robotic manipulation tasks10.1016/j.rcim.2022.102517 ↗
5. Deep learning for high-dimensional reliability analysis10.1016/j.ymssp.2019.106399 ↗
6. Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation
7. The movielens datasets: History and context10.1145/2827872 ↗
8. Personalized movie recommendation system combining data mining with the k-clique method
9. Peng, S., Park, D.S., Kim, D.Y., Yang, Y., Siet, S., Ugli SI, R., and Lee, H. (2022). International Conference on Computer Science and Its Applications and the International Conference on Ubiquitous Information Technologies and Applications, Springer Nature Singapore.
10. Koren, Y., Rendle, S., and Bell, R. (2021). Recommender Systems Handbook, Springer.
11. Using collaborative filtering to weave an information tapestry10.1145/138859.138867 ↗
12. Matrix factorization techniques for recommender systems10.1109/MC.2009.263 ↗
13. Liang, D., Altosaar, J., Charlin, L., and Blei, D.M. (2016, January 15–19). Factorization meets the item embedding: Regularizing matrix factorization with item co-occurrence. Proceedings of the 10th ACM Conference on Recommender Systems, Boston, MA, USA.10.1145/2959100.2959182 ↗
14. Tran, T., Lee, K., Liao, Y., and Lee, D. (2018, January 22–26). Regularizing matrix factorization with user and item embeddings for recommendation. Proceedings of the 27th ACM International Conference on Information and Knowledge Management, Torino, Italy.10.1145/3269206.3271730 ↗
15. Movie genome: Alleviating new item cold start in movie recommendation10.1007/s11257-019-09221-y ↗
16. Movie recommendation algorithm using social network analysis to alleviate cold-start problem
17. An efficient movie recommendation algorithm based on improved k-clique10.1186/s13673-018-0161-6 ↗
18. Van Meteren, R., and Van Someren, M. (2000, January 30). Using content-based filtering for recommendation. Proceedings of the Machine Learning in the New Information Age: MLnet/ECML2000 Workshop, Barcelona, Spain.
19. Semantic audio content-based music recommendation and visualization based on user preference examples10.1016/j.ipm.2012.06.004 ↗
20. Li, L., Wang, D., Li, T., Knox, D., and Padmanabhan, B. (2011, January 24–28). Scene: A scalable two-stage personalized news recommendation system. Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, Beijing, China.10.1145/2009916.2009937 ↗
21. College library personalized recommendation system based on hybrid recommendation algorithm10.1016/j.procir.2019.04.126 ↗
22. Wang, H., Wang, N., and Yeung, D.Y. (2014, January 10–13). Collaborative deep learning for recommender systems. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia.10.1145/2783258.2783273 ↗
23. Deep learning based recommender system: A survey and new perspectives10.1145/3158369 ↗
24. Cheng, H.T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., and Ispir, M. (2016, January 15). Wide & deep learning for recommender systems. Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, Boston, MA, USA.10.1145/2988450.2988454 ↗
25. Naumov, M., Mudigere, D., Shi, H.J.M., Huang, J., Sundaraman, N., Park, J., Wang, X., Gupta, U., Wu, C.-J., and Azzolini, A.G. (2019). Deep learning recommendation model for personalization and recommendation systems. arXiv.
26. Li, Z., Shi, L., Cristea, A.I., and Zhou, Y. (July, January 28). A survey of collaborative reinforcement learning: Interactive methods and design patterns. Proceedings of the 2021 ACM Designing Interactive Systems Conference, Virtual.10.1145/3461778.3462135 ↗
27. Zhao, X., Xia, L., Zou, L., Yin, D., and Tang, J. (2019). Toward simulating environments in reinforcement learning based recommendations. arXiv.10.1145/3240323.3240374 ↗
28. Deliu, N. (2023). Reinforcement learning for sequential decision making in population research. Qual. Quant., 1–24.10.1007/s11135-023-01755-z ↗
29. Zou, L., Xia, L., Ding, Z., Song, J., Liu, W., and Yin, D. (2019, January 4–8). Reinforcement learning to optimize long-term user engagement in recommender systems. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.10.1145/3292500.3330668 ↗
30. Proposed model to intelligent recommendation system based on Markov chains and grouping of genres10.1016/j.procs.2020.09.082 ↗
31. Deep reinforcement learning: A brief survey10.1109/MSP.2017.2743240 ↗
32. Surprise: A Python library for recommender systems10.21105/joss.02174 ↗
33. Cornac: A comparative framework for multimodal recommender systems