Two-Layer Matrix Factorization and Multi-Layer Perceptron for Online Service Recommendation
Source abstract
Abstract
Service recommendation is key to improving users’ online experience. The development of the Internet has accelerated the creation of many services, and whether users can obtain good experiences among the massive number of services mainly depends on the quality of service recommendation. It is commonly believed that deep learning has excellent nonlinear fitting ability in capturing the complex interactions between users and items. The advantage in learning intricacy relationships enables deep learning to become an important technology for present service recommendation. Recently, it is noticed that linear models can perform almost as well as the state-of-the-art deep learning models, suggesting that capturing linear relationships between users and items is also very important for recommender systems. Therefore, numerous deep learning systems combined with linear models have been proposed. However, existing models are incapable of considering the size of the embedding. When the embedding dimension is too large, it leads to overfitting and thus influences the model’s ability to capture linear relationships. In this paper, a neural network based on two-layer matrix factorization and multi-layer perceptron—Two-layer Matrix factorization and Multi-layer perceptron Neural Network (TMMNN)—is proposed. To solve the problem of overfitting caused by an oversized embedding dimension, multi-size embedding technology has been integrated into the model. Matrix factorization and the multi-layer perceptron are placed in the upper and lower layers respectively, and they both receive embedding vectors dynamically adjusted for dimensions. In the upper layer, the matrix factorization is responsible for receiving the embedding of users and items, capturing linear relationships, and then yielding the generated new vectors as input to the multi-layer perceptron in the lower layer. Compared to other previously proposed models, the experimental results on the standard datasets MovieLens 20M and MovieLens Latest show that the TMMNN model is evidently better in terms of prediction accuracy.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Recommender Systems and Techniques · Image Retrieval and Classification Techniques · Sentiment Analysis and Opinion Mining
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Engineering, Ningbo University of Technology, Ningbo 315211, China
College of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210046, China
Southwest Institute of Electronic Technology, Chengdu 610036, China
College of Computer Science, Tsinghua University, Beijing 100084, China
School of Computing and Information Technology, University of Wollongong, Wollongong, NSW 2522, Australia
References (22)
Show source-provided references
1. Deep learning based recommender system: A survey and new perspectives10.1145/3158369 ↗
2. Cheng, H.T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., and Ispir, M. (2016, January 15). Wide and deep learning for recommender systems. Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, Boston, MA, USA.10.1145/2988450.2988454 ↗
3. Wang, R., Fu, B., Fu, G., and Wang, M. (2017, January 13–17). Deep and cross network for ad click predictions. Proceedings of the ADKDD’17, Halifax, NS, Canada.10.1145/3124749.3124754 ↗
4. Liu, H., Zhao, X., Wang, C., Liu, X., and Tang, J. (2020, January 25–30). Automated embedding size search in deep recommender systems. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Online.10.1145/3397271.3401436 ↗
5. Liu, B., Tang, R., Chen, Y., Yu, J., Guo, H., and Zhang, Y. (2019, January 13–17). Feature generation by convolutional neural network for click-through rate prediction. Proceedings of the World Wide Web Conference, San Francisco, CA, USA.10.1145/3308558.3313497 ↗
6. Rendle, S., Krichene, W., Zhang, L., and Anderson, J. (2020, January 22–26). Neural collaborative filtering vs. matrix factorization revisited. Proceedings of the Fourteenth ACM Conference on Recommender Systems, Online.10.1145/3383313.3412488 ↗
7. Andoni, A., Panigrahy, R., Valiant, G., and Zhang, L. (2014, January 21–26). Learning polynomials with neural networks. Proceedings of the International Conference on Machine Learning, Beijing, China.
8. Guo, H., Tang, R., Ye, Y., Li, Z., and He, X. (2017). DeepFM: A factorization-machine based neural network for CTR prediction. arXiv.10.24963/ijcai.2017/239 ↗
9. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. Proceedings of the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016.10.1109/CVPR.2016.90 ↗
10. Matrix factorization techniques for recommender systems10.1109/MC.2009.263 ↗
11. Jin, R., Li, D., Gao, J., Liu, Z., Chen, L., and Zhou, Y. (2021, January 14–18). Towards a better understanding of linear models for recommendation. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Online.10.1145/3447548.3467428 ↗
12. Steck, H. (2019, January 13–17). Embarrassingly shallow autoencoders for sparse data. Proceedings of the World Wide Web Conference, San Francisco, CA, USA.10.1145/3308558.3313710 ↗
13. Rendle, S. (2010, January 13–17). Factorization machines. Proceedings of the 2010 IEEE International Conference on Data Mining, Sydney, Australia.10.1109/ICDM.2010.127 ↗
14. Juan, Y., Zhuang, Y., Chin, W.S., and Lin, C.J. (2016, January 15–19). Field-aware factorization machines for CTR prediction. Proceedings of the 10th ACM Conference on Recommender Systems, Boston, MA, USA.10.1145/2959100.2959134 ↗
15. Lian, J., Zhou, X., Zhang, F., Chen, Z., Xie, X., and Sun, G. (2018, January 19–23). xdeepfm: Combining explicit and implicit feature interactions for recommender systems. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, London, UK.10.1145/3219819.3220023 ↗
16. Elkahky, A.M., Song, Y., and He, X. (2015, January 18–22). A multi-view deep learning approach for cross domain user modeling in recommendation systems. Proceedings of the 24th International Conference on World Wide Web, Florence, Italy.10.1145/2736277.2741667 ↗
17. Qu, Y., Cai, H., Ren, K., Zhang, W., Yu, Y., Wen, Y., and Wang, J. (2016, January 12–15). Product-based neural networks for user response prediction. Proceedings of the 2016 IEEE 16th International Conference on Data Mining (ICDM), Barcelona, Spain.10.1109/ICDM.2016.0151 ↗
18. Liu, H., Simonyan, K., and Yang, Y. (2018). Darts: Differentiable architecture search. arXiv.
19. Zhao, X., Liu, H., Fan, W., Liu, H., Tang, J., Wang, C., Chen, M., Zheng, X., Liu, X., and Yang, X. (2021, January 7–10). AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations. Proceedings of the 2021 IEEE International Conference on Data Mining (ICDM), Auckland, New Zealand.10.1109/ICDM51629.2021.00101 ↗
20. He, X., Liao, L., Zhang, H., Nie, L., Hu, X., and Chua, T.S. (2017, January 3–7). Neural collaborative filtering. Proceedings of the 26th International Conference on World Wide Web, Perth, Australia.10.1145/3038912.3052569 ↗
21. The MovieLens Datasets: History and Context10.1145/2827872 ↗
22. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.