The impact of visual emotional cues in cultural heritage on public sentiment and behavioral intention: an image emotion recognition approach
Source abstract
Abstract
Abstract This study developed the Heritage Sentiment Index (HSI) using deep learning to detect emotions in cultural heritage images on social media. By analyzing images of historical sites, intangible heritage, and artifacts, this research created a daily sentiment series linked to tourism intentions and user interactions. Results show that HSI accurately predicts public emotional responses and offline participation, especially during sensitive periods such as cultural disputes and disasters. Comparing image-based sentiment (HSI) with comment-based sentiment (CSI) reveals a dual-path process: images prompt immediate reactions, while comments influence delayed responses. When aligned, they amplify emotions; when competing, substitution occurs. This research highlights the emotional impact of heritage imagery in cultural tourism, offering insights for improving digital heritage experiences, multimodal content design, and visitor engagement strategies.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Diverse Aspects of Tourism Research · Aesthetic Perception and Analysis · Visual Attention and Saliency Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (54)
Show source-provided references
1. Karl, M., Kock, F., Ritchie, B. W. & Gauss, J. Affective forecasting and travel decision-making: An investigation in times of a pandemic. Ann. Tour. Res. 87, 103139. https://doi.org/10.1016/j.annals.2021.103139 (2021).10.1016/j.annals.2021.103139 ↗
2. Naseer, K. et al. Travel behaviour prediction amid COVID-19 underlaying situational awareness theory and health belief model. Behav. Inf. Technol. 41, 3318–3328 (2022).10.1080/0144929X.2021.1984579 ↗
3. Neidhardt, J., Rümmele, N. & Werthner, H. Predicting happiness: user interactions and sentiment analysis in an online travel forum. Inf. Technol. Tour. 17, 101–119 (2017).10.1007/s40558-017-0079-2 ↗
4. Fu, X., Liu, X. & Li, Z. Catching eyes of social media wanderers: How pictorial and textual cues in visitor-generated content shape users’ cognitive-affective psychology. Tour. Manag. 100, 104815 (2024).10.1016/j.tourman.2023.104815 ↗
5. Xu, H. et al. Understanding the influence of user-generated content on tourist loyalty behavior in a cultural World Heritage Site. Tour. Recreat. Res. 48, 173–187 (2023).
6. Su, D. N., Nguyen, N. A. N., Nguyen, Q. N. T. & Tran, T. P. The link between travel motivation and satisfaction towards a heritage destination: The role of visitor engagement, visitor experience and heritage destination image. Tour. Manag. Perspect. 34, 100634 (2020).
7. Cominelli, F. & Greffe, X. Intangible cultural heritage: Safeguarding for creativity. City, Cult. Soc. 3, 245–250 (2012).10.1016/j.ccs.2012.10.003 ↗
8. Kätsyri, J., Ravaja, N. & Salminen, M. Aesthetic images modulate emotional responses to reading news messages on a small screen: A psychophysiological investigation. Int. J. Hum.-Comput. Stud. 70, 72–87 (2012).10.1016/j.ijhcs.2011.08.007 ↗
9. Birjali, M., Kasri, M. & Beni-Hssane, A. A comprehensive survey on sentiment analysis: Approaches, challenges and trends. Knowl.-Based Syst. 226, 107134 (2021).10.1016/j.knosys.2021.107134 ↗
10. Zhu, L., Zhu, Z., Zhang, C., Xu, Y. & Kong, X. Multimodal sentiment analysis based on fusion methods: A survey. Inf. Fusion 95, 306–325 (2023).10.1016/j.inffus.2023.02.028 ↗
11. Fiorucci, M. et al. Machine learning for cultural heritage: a survey. Pattern Recognit. Lett. 133, 102–108 (2020).10.1016/j.patrec.2020.02.017 ↗
12. Gîrbacia, F. An Analysis of Research Trends for Using Artificial Intelligence in Cultural Heritage. Electronics 13, 3738 (2024).10.3390/electronics13183738 ↗
13. Furini, M., Mandreoli, F., Martoglia, R. & Montangero, M. A predictive method to improve the effectiveness of twitter communication in a cultural heritage scenario. ACM J. Comput. Cult. Herit. (JOCCH) 15, 1–18 (2022).10.1145/3470786 ↗
14. Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 84–90 (2017).10.1145/3065386 ↗
15. Song, J., Gao, S., Zhu, Y. & Ma, C. A survey of remote sensing image classification based on CNNs. Big Earth Data 3, 232–254 (2019).10.1080/20964471.2019.1657720 ↗
16. Zou, Z., Zhao, X., Zhao, P., Qi, F. & Wang, N. CNN-based statistics and location estimation of missing components in routine inspection of historic buildings. J. Cult. Herit. 38, 221–230 (2019).10.1016/j.culher.2019.02.002 ↗
17. Dufitimana, E. et al. Measuring urban socio-economic disparities in the global south from space using convolutional neural network: the case of the City of Kigali, Rwanda. GeoJournal 89, 107 (2024).10.1007/s10708-024-11122-6 ↗
18. You, Q., Luo, J., Jin, H. & Yang, J. Robust image sentiment analysis using progressively trained and domain transferred deep networks. In Proceedings of the AAAI Conference on Artificial Intelligence. 29. https://doi.org/10.1609/aaai.v29i1.9179. (2015).10.1609/aaai.v29i1.9179 ↗
19. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. & Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; pp 2818–2826 (2016).10.1109/CVPR.2016.308 ↗
20. Sam, S. M., Kamardin, K., Sjarif, N. N. A. & Mohamed, N. Offline signature verification using deep learning convolutional neural network (CNN) architectures GoogLeNet inception-v1 and inception-v3. Procedia Comput. Sci. 161, 475–483 (2019).10.1016/j.procs.2019.11.147 ↗
21. Yang, L., Hanneke, S. & Carbonell, J. A theory of transfer learning with applications to active learning. Mach. Learn. 90, 161–189 (2013).10.1007/s10994-012-5310-y ↗
22. Loughran, T. & McDonald, B. When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks. J. Financ. 66, 35–65 (2011).10.1111/j.1540-6261.2010.01625.x ↗
23. Barrett, L. F. Categories and their role in the science of emotion. Psychol. Inq. 28, 20–26 (2017).10.1080/1047840X.2017.1261581 ↗
24. Tetlock, P. C. Giving content to investor sentiment: The role of media in the stock market. J. Financ. 62, 1139–1168 (2007).10.1111/j.1540-6261.2007.01232.x ↗
25. Amir, A., Farach, M., Idury, R. M., Lapoutre, J. A. & Schaffer, A. A. Improved dynamic dictionary matching. Inf. Comput. 119, 258–282 (1995).10.1006/inco.1995.1090 ↗
26. Song, M. Chambers, T. Text mining with the Stanford CoreNLP. In Measuring Scholarly Impact: Methods And Practice, Springer; pp 215–234 (2014).10.1007/978-3-319-10377-8_10 ↗
27. Zhang, S. et al. Deep learning-based multimodal emotion recognition from audio, visual, and text modalities: A systematic review of recent advancements and future prospects. Expert Syst. Appl. 237, 121692 (2024).10.1016/j.eswa.2023.121692 ↗
28. Ezzameli, K. & Mahersia, H. Emotion recognition from unimodal to multimodal analysis: A review. Inf. Fusion 99, 101847 (2023).10.1016/j.inffus.2023.101847 ↗
29. Socher, R. et al. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods In Natural Language Processing; pp 1631–1642 (2013).10.18653/v1/D13-1170 ↗
30. Dixon, W. J. & Yuen, K. K. Trimming and winsorization: A review. Stat. Hefte 15, 157–170 (1974).10.1007/BF02922904 ↗
31. Wang, Y. et al. A systematic review on affective computing: Emotion models, databases, and recent advances. Inf. Fusion 83, 19–52 (2022).10.1016/j.inffus.2022.03.009 ↗
32. Kratzwald, B., Ilić, S., Kraus, M., Feuerriegel, S. & Prendinger, H. Deep learning for affective computing: Text-based emotion recognition in decision support. Decis. Support Syst. 115, 24–35 (2018).10.1016/j.dss.2018.09.002 ↗
33. Lucas, J. The forgotten movement to reclaim Africa’s stolen art. New Yorker, 14 (2022).
34. Rivera Mindt, M., Byrd, D., Saez, P. & Manly, J. Increasing culturally competent neuropsychological services for ethnic minority populations: A call to action. Clin. Neuropsychol. 24, 429–453 (2010).10.1080/13854040903058960 ↗
35. Kucukergin, K. G. & Uygur, S. M. Are emotions contagious? Developing a destination social servicescape model. J. Destin. Mark. Manag. 14, 100386 (2019).
36. Stamkou, E., Keltner, D., Corona, R., Aksoy, E. & Cowen, A. S. Emotional palette: a computational mapping of aesthetic experiences evoked by visual art. Sci. Rep. 14, 19932 (2024).10.1038/s41598-024-69686-9 ↗
37. Lin, R. et al. Visualized Emotion Ontology: a model for representing visual cues of emotions. BMC Med. Inform. Decis. Mak. 18, 101–113 (2018).10.1186/s12911-018-0634-6 ↗
38. Uhrig, M. K. et al. Emotion elicitation: A comparison of pictures and films. Front. Psychol. 7, 180 (2016).10.3389/fpsyg.2016.00180 ↗
39. Lindquist, K. A., Jackson, J. C., Leshin, J., Satpute, A. B. & Gendron, M. The cultural evolution of emotion. Nat. Rev. Psychol. 1, 669–681 (2022).10.1038/s44159-022-00105-4 ↗
40. Masuda, T. Culture and attention: Recent empirical findings and new directions in cultural psychology. Soc. Personal. Psychol. Compass 11, e12363 (2017).10.1111/spc3.12363 ↗
41. Karlsson, N., Loewenstein, G. & Seppi, D. The ostrich effect: Selective attention to information. J. Risk Uncertain. 38, 95–115 (2009).10.1007/s11166-009-9060-6 ↗
42. King, L. M. & Halpenny, E. A. Communicating the World Heritage brand: visitor awareness of UNESCO’s World Heritage symbol and the implications for sites, stakeholders and sustainable management. J. Sustain. Tour. 22, 768–786 (2014).10.1080/09669582.2013.864660 ↗
43. Newey, W. K. & West, K. D. A Simple, Positive Semi-definite, Heteroskedasticity and Autocorrelationconsistent Covariance Matrix, (1986).10.3386/t0055 ↗
44. Aiken, L. S. Multiple Regression: Testing and Interpreting Interactions; Sage, (1991).
45. Matsumoto, D. & Hwang, H. S. Culture and emotion: The integration of biological and cultural contributions. J. Cross-Cult. Psychol. 43, 91–118 (2012).10.1177/0022022111420147 ↗
46. Maki, R. H. Predicting performance on text: Delayed versus immediate predictions and tests. Mem. Cogn. 26, 959–964 (1998).10.3758/BF03201176 ↗
47. Li, Q., Huang, Z. J. & Christianson, K. Visual attention toward tourism photographs with text: An eye-tracking study. Tour. Manag. 54, 243–258 (2016).10.1016/j.tourman.2015.11.017 ↗
48. Bakar, M. H. A., Desa, M. A. M. & Mustafa, M. Attributes for image content that attract consumers’ attention to advertisements. Procedia-Soc. Behav. Sci. 195, 309–314 (2015).10.1016/j.sbspro.2015.06.349 ↗
49. Lenski, S. & Großschedl, J. Emotional design pictures: Pleasant but too weak to evoke arousal and attract attention? Front. Psychol. 13, 966287 (2023).10.3389/fpsyg.2022.966287 ↗
50. Kaplan, E. A. Global trauma and public feelings: Viewing images of catastrophe. Consum., Mark. Cult. 11, 3–24 (2008).10.1080/10253860701799918 ↗
51. Meek, A. Trauma and Media: Theories, Histories, and Images; Routledge. https://doi.org/10.4324/9780203863190. (2011).10.4324/9780203863190 ↗
52. Filip, A. M. & Pochea, M. M. Intentional and spurious herding behavior: A sentiment driven analysis. J. Behav. Exp. Financ. 38, 100810 (2023).10.1016/j.jbef.2023.100810 ↗
53. Bufquin, D., Park, J.-Y., Back, R. M., Nutta, M. W. & Zhang, T. Effects of hotel website photographs and length of textual descriptions on viewers’ emotions and behavioral intentions. Int. J. Hosp. Manag. 87, 102378 (2020).10.1016/j.ijhm.2019.102378 ↗
54. Orth, U. et al. Effect size guidelines for cross-lagged effects. Psychol. Methods 29, 421 (2024).10.1037/met0000499 ↗