Improved Lightweight Multi-Target Recognition Model for Live Streaming Scenes
Source abstract
Abstract
Nowadays, the commercial potential of live e-commerce is being continuously explored, and machine vision algorithms are gradually attracting the attention of marketers and researchers. During live streaming, the visuals can be effectively captured by algorithms, thereby providing additional data support. This paper aims to consider the diversity of live streaming devices and proposes an extremely lightweight and high-precision model to meet different requirements in live streaming scenarios. Building upon yolov5s, we incorporate the MobileNetV3 module and the CA attention mechanism to optimize the model. Furthermore, we construct a multi-object dataset specific to live streaming scenarios, including anchor facial expressions and commodities. A series of experiments have demonstrated that our model realized a 0.4% improvement in accuracy compared to the original model, while reducing its weight to 10.52%.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Image and Video Quality Assessment · Visual Attention and Saliency Detection · Face recognition and analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Economics and Management, Shanghai Institute of Technology, Shanghai 200235, China
School of Cultural Heritage and Information Management, Shanghai University, Shanghai 200444, China
Business School, East China University of Science and Technology, Shanghai 200237, China
Funding
National Natural Science Foundation of China
References (43)
Show source-provided references
1. What motivates users’ viewing and purchasing behavior motivations in live streaming: A stream-streamer-viewer perspective10.1016/j.jretconser.2022.103240 ↗
2. “Oh, My God, Buy It!” Investigating impulse buying behavior in live streaming commerce10.1080/10447318.2022.2076773 ↗
3. Morris, T. (2004). Computer Vision and Image Processing, Palgrave Macmillan Ltd.
4. Exploring deep learning-based architecture, strategies, applications and current trends in generic object detection: A comprehensive review10.1109/ACCESS.2020.3021508 ↗
5. Object detection using YOLO: Challenges, architectural successors, datasets and applications10.1007/s11042-022-13644-y ↗
6. Attention mechanisms in computer vision: A survey10.1007/s41095-022-0271-y ↗
7. Bazarevsky, V., Kartynnik, Y., Vakunov, A., Raveendran, K., and Grundmann, M. (2019). Blazeface: Sub-millisecond neural face detection on mobile gpus. arXiv.
8. Jin, R., Xu, Y., Xue, W., Li, B., Yang, Y., and Chen, W. (2021). International Conference on Advanced Hybrid Information Processing, Springer International Publishing.
9. An improved YOLOv5 model based on visual attention mechanism: Application to recognition of tomato virus disease10.1016/j.compag.2022.106780 ↗
10. Xu, S., Guo, Z., Liu, Y., Fan, J., and Liu, X. (2022). Artificial Neural Networks and Machine Learning–ICANN 2022, Proceedings of the 31st International Conference on Artificial Neural Networks, Bristol, UK, 6–9 September 2022, Part III, Springer Nature.
11. Research on efficient feature extraction: Improving YOLOv5 backbone for facial expression detection in live streaming scenes10.3389/fncom.2022.980063 ↗
12. Affective causes and consequences of social information processing
13. The effect of the expressed anger and sadness on online news believability10.1080/07421222.2021.1990607 ↗
14. A new livestream retail analytics framework to assess the sales impact of emotional displays10.1177/00222429211013042 ↗
15. Happiness begets money: Emotion and engagement in live streaming10.1177/00222437211002477 ↗
16. An integrative review of sensory marketing: Engaging the senses to affect perception, judgment and behavior10.1016/j.jcps.2011.08.003 ↗
17. Mood states and consumer behavior: A critical review10.1086/208516 ↗
18. The influence of positive affect on variety seeking among safe, enjoyable products10.1086/209347 ↗
19. Ng, H.W., Nguyen, V.D., Vonikakis, V., and Winkler, S. (2015, January 9–13). Deep learning for emotion recognition on small datasets using transfer learning. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, Seattle, WA, USA.10.1145/2818346.2830593 ↗
20. Abbaschian, B.J., Sierra-Sosa, D., and Elmaghraby, A. (2021). Deep learning techniques for speech emotion recognition, from databases to models. Sensors, 21.10.3390/s21041249 ↗
21. The ripple effect: Emotional contagion and its influence on group behavior10.2307/3094912 ↗
22. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
23. Faster R-CNN: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
24. Glenn, J. (2023, March 04). yolov5. Git Code. Available online: https://github.com/ultralytics/yolov5.
25. Channel pruned YOLO V5s-based deep learning approach for rapid and accurate apple fruitlet detection before fruit thinning10.1016/j.biosystemseng.2021.08.015 ↗
26. A pavement distresses identification method optimized for YOLOv5s10.1038/s41598-022-07527-3 ↗
27. Research on Energy Efficiency Management of Forklift Based on Improved YOLOv5 Algorithm10.1155/2021/5808221 ↗
28. A review on the attention mechanism of deep learning10.1016/j.neucom.2021.03.091 ↗
29. Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, K. (2015, January 7–12). Spatial transformer networks. Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, QC, Canada.
30. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
31. Parallel deep learning algorithms with hybrid attention mechanism for image segmentation of lung tumors10.1109/TII.2020.3022912 ↗
32. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
33. Hou, Q., Zhou, D., and Feng, J. (2021, January 20–25). Coordinate attention for efficient mobile network design. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01350 ↗
34. Zhang, Q.L., and Yang, Y.B. (2021, January 6–11). Sa-net: Shuffle attention for deep convolutional neural networks. Proceedings of the ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9414568 ↗
35. Pantic, M., Valstar, M., Rademaker, R., and Maat, L. (2005, January 6–8). Web-based database for facial expression analysis. Proceedings of the IEEE International Conference on Multimedia and Expo, Amsterdam, The Netherlands.
36. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I. (2010, January 13–18). The extended Cohn–Kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, San Francisco, CA, USA.10.1109/CVPRW.2010.5543262 ↗
37. Positive emotion bias: Role of emotional content from online customer reviews in purchase decisions10.1016/j.jretconser.2019.101891 ↗
38. Measuring the hedonic and utilitarian dimensions of consumer attitude10.1509/jmkr.40.3.310.19238 ↗
39. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.
40. Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (November, January 10). Searching for MobileNetV3. Proceedings of the IEEE/CVF international conference on computer vision, Seoul, Republic of Korea.
41. Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., and Le, Q.V. (2019, January 15–20). Mnasnet: Platform-aware neural architecture search for mobile. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00293 ↗
42. Borisyuk, F., Gordo, A., and Sivakumar, V. (2018, January 19–23). Rosetta: Large scale system for text detection and recognition in images. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.10.1145/3219819.3219861 ↗
43. Benchmark analysis of representative deep neural network architectures10.1109/ACCESS.2018.2877890 ↗