Proposal and Evaluation of a Robot to Improve the Cognitive Abilities of Novice Baseball Spectators Using a Method for Selecting Utterances Based on the Game Situation
Source abstract
Abstract
Herein, an engineering method is developed to improve spectators’ sports-watching abilities. We present an interactive robot that supports the cognitive abilities of baseball novices in sports spectating. The robot watches a game with a person and utters words applicable to the game situation. We propose two methods to support cognitive watching: five categories of utterances (confirmation, information, prediction, emotion, and evaluation) and utterance rules for player participation and game scenes. We also propose a method for generating utterances for each category. To evaluate the generated utterances, we conducted an experiment whereby spectators watched baseball footage with the robot. The results of the experiment showed that the robot’s utterances could support the cognitive ability sub-factor of individual game intelligence, specifically “Spectating while knowing the player’s strengths.” In addition, the feeling of heaviness that occurs when watching alone could potentially be reduced by watching with a robot. This study is the first to attempt to support and improve spectators’ abilities to watch sports using the human–robot interaction approach. We anticipate that this approach will be used in the future to build a relationship (increase intimacy) with an agent (robot) and to support and improve cognitive abilities regardless of the type of sport.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Analysis and Summarization · Speech and dialogue systems · Human Motion and Animation
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Graduate School of Engineering, Tokyo Polytechnic University, Atsugi 243-0213, Japan
Graduate School of Informatics and Engineering, The University of Electro-Communications, Chofu 182-8585, Japan
References (39)
Show source-provided references
1. A conceptual work-frame of spectator behavior: From a management point of view of cultural creativity of spectator sports
2. Explication of cognitive capability on sports spectating
3. EAR: Enhanced augmented reality system for sports entertainment applications
4. The impact of virtual reality (VR) technology on sport spectators’ flow experience and satisfaction10.1016/j.chb.2018.12.040 ↗
5. TV-watching robot: Toward enriching media experience and activating human communication10.5594/JMI.2022.3160804 ↗
6. Zhou, J., and Salvendy, G. (2019). Atmosphere Sharing with TV Chat Agents for Increase of User’s Motivation for Conversation. Human Aspects of IT for the Aged Population. Design for the Elderly and Technology Acceptance, Springer International Publishing.
7. Yamamoto, F., Ayedoun, E., and Tokumaru, M. (2022, January 29). Human-robot interaction environment to enhance the sense of presence in remote sports watching. Proceedings of the 2022 Joint 12th International Conference on Soft Computing and Intelligent Systems and 23rd International Symposium on Advanced Intelligent Systems (SCIS & ISIS), Ise, Japan.10.1109/SCISISIS55246.2022.10002053 ↗
8. How do sports viewers watch a game? A trial mainly on conversation analysis
9. A video-based 3D-reconstruction of soccer games10.1111/1467-8659.00431 ↗
10. Koyama, T., Kitahara, I., and Ohta, Y. (2003, January 7–10). Live mixed-reality 3D video in soccer stadium. Proceedings of the 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality, Tokyo, Japan.10.1145/965400.965463 ↗
11. Inamoto, N., and Saito, H. (2004, January 3–5). Free viewpoint video synthesis and presentation of sporting events for mixed reality entertainment. Proceedings of the 2004 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, Singapore.10.1145/1067343.1067348 ↗
12. Matsui, K., Iwase, M., Agata, M., Tanaka, T., and Ohnishi, N. (1998, January 25). Soccer image sequence computed by a virtual camera. Proceedings of the 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, USA.
13. Liang, D., Liu, Y., Huang, Q., Zhu, G., Jiang, S., Zhang, Z., and Gao, W. (2005, January 6–11). Video2Cartoon: Generating 3D cartoon from broadcast soccer video. Proceedings of the 13th ACM International Conference on Multimedia, Singapore.10.1145/1101149.1101184 ↗
14. Interactive augmented reality in digital broadcasting environments
15. A mixed-reality system for broadcasting sports video to mobile devices10.1109/MMUL.2010.24 ↗
16. Lee, S.-O., Ahn, S.C., Hwang, J.-I., and Kim, H.-G. (2011, January 9–14). A vision-based mobile augmented reality system for baseball games. Proceedings of the Virtual and Mixed Reality—New Trends—International Conference, Virtual and Mixed Reality, Orlando, FL, USA.10.1007/978-3-642-22021-0_8 ↗
17. Bielli, S., and Harris, C.G. (2015, January 9–11). A mobile augmented reality system to enhance live sporting events. Proceedings of the 6th Augmented Human International Conference, Singapore.10.1145/2735711.2735836 ↗
18. Mizushina, Y., Fujiwara, W., Sudou, T., Fernando, C.L., Minamizawa, K., and Tachi, S. (2015, January 9–11). Interactive instant replay: Sharing sports experience using 360-degrees spherical images and haptic sensation based on the coupled body motion. Proceedings of the 6th Augmented Human International Conference, Singapore.10.1145/2735711.2735778 ↗
19. Exploring the sport spectator experience: Virtual football spectatorship in the pub 110.1080/14660970701811057 ↗
20. Tanaka, K., Nakashima, H., Noda, I., and Hasida, K. (1998, January 3–7). MIKE: An automatic commentary system for soccer. Proceedings of the International Conference on Multi Agent Systems, Paris, France.
21. Tanaka-Ishii, K., Hasida, K., and Noda, I. (1998, January 10). Reactive content selection in the generation of real-time soccer commentary. Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics—Volume 2, Montreal, QC, Canada.10.3115/980691.980778 ↗
22. Chen, D.L., and Mooney, R.J. (2008, January 5–9). Learning to sportscast: A test of grounded language acquisition. Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland.10.1145/1390156.1390173 ↗
23. Automated event recognition for football commentary generation10.4018/jgcms.2010100105 ↗
24. Generating live soccer-match commentary from play data
25. Kumano, T., Ichiki, M., Kurihara, K., Kaneko, H., Komori, T., Shimizu, T., Seiyama, N., Imai, A., Sumiyoshi, H., and Takagi, T. (2019, January 5–7). Generation of automated sports commentary from live sports data. Proceedings of the 2019 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Jeju, Republic of Korea.10.1109/BMSB47279.2019.8971879 ↗
26. Kim, B.J., and Choi, Y.S. (2020, January 30). Automatic baseball commentary generation using deep learning. Proceedings of the 35th Annual ACM Symposium on Applied Computing, Brno, Czech Republic.10.1145/3341105.3374063 ↗
27. Chan, C.-Y., Hui, C.-C., Siu, W.-C., Chan, S.-W., and Chan, H.A. (2022, January 1–4). To start automatic commentary of soccer game with mixed spatial and temporal attention. Proceedings of the TENCON 2022 IEEE Region 10 Conference (TENCON), Hong Kong, Hong Kong.10.1109/TENCON55691.2022.9978078 ↗
28. Sentence-style conversion for text-to-speech application
29. Takami, Y. (2020). [Multiview Tactical Diagrams: How to Win in Baseball by Understanding the Right Theories and Controlling the Pitcher-Batter Duel] Maruchi Anguru Senjutsu Zukai Yakyuu no Tatakai Kata Tadashii Seorii Wo Rikai Shite “Toushu Tai Dasha” Wo Seisuru, Baseball Magazine Sha. (In Japanese).
30. Thakur, N. (2023). Social Media Mining and Analysis: A Brief Review of Recent Challenges. Information, 14.10.3390/info14090484 ↗
31. Mining social media for newsgathering: A Review10.1016/j.osnem.2019.100049 ↗
32. Sugimori, M., Mizuno, M., and Sasahara, K. (2019, January 4). A social media analysis of collective enthusiasm in Japanese professional baseball. Proceedings of the 33rd Annual Conference of the Japanese Society for Artificial Intelligence, Niigata, Japan.
33. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019, January 2–7). BERT: Pretraining of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.
34. (2023, March 01). GitHub—Cl-Tohoku_Bert-Japanese_BERT Models for Japanese Text. Available online: https://github.com/cl-tohoku/bert-japanese.
35. Simulating speech systems10.1016/0885-2308(91)90019-M ↗
36. Emotion in sport spectator behavior: Development of a measurement scale and its application to models
37. Repurchase behavior at sporting events: Beyond repurchase intentions10.5225/jjsm.2013-001 ↗
38. On a test of whether one of two random variables is stochastically larger than the other10.1214/aoms/1177730491 ↗
39. Individual comparisons by ranking methods10.2307/3001968 ↗