journal article·2025-07-03Open access

Decoding Humor-Induced Amusement via Facial Expression Analysis: Toward Emotion-Aware Applications

Gabrielle Toupin, Arthur Dehgan, Marie Buffo, Clément Feyt, Golnoush Alamian, Karim Jerbi, Anne-Lise Saive

Applied Sciences· Volume 15· Issue 13

Abstract

Humor is widely recognized for its positive effects on well-being, including stress reduction, mood enhancement, and cognitive benefits. Yet, the lack of reliable tools to objectively quantify amusement—particularly its temporal dynamics—has limited progress in this area. Existing measures often rely on self-report or coarse summary ratings, providing little insight into how amusement unfolds over time. To address this gap, we developed a Random Forest model to predict the intensity of amusement evoked by humorous video clips, based on participants’ facial expressions—particularly the co-activation of Facial Action Units 6 and 12 (“% Smile”)—and video features such as motion, saliency, and topic. Our results show that exposure to humorous content significantly increases “% Smile”, with amusement peaking toward the end of videos. Importantly, we observed emotional carry-over effects, suggesting that consecutive humorous stimuli can sustain or amplify positive emotional responses. Even when trained solely on humorous content, the model reliably predicted amusement intensity, underscoring the robustness of our approach. Overall, this study provides a novel, objective method to track amusement on a fine temporal scale, advancing the measurement of nonverbal emotional expression. These findings may inform the design of emotion-aware applications and humor-based therapeutic interventions to promote well-being and emotional health.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Humor Studies and Applications · Video Analysis and Summarization

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Cognitive and Computational Neuroscience Lab, Psychology Department, University of Montréal, Montreal, QC H3T 1J4, Canada

Independent Scientist, 38000 Grenoble, France

Mila, The Quebec AI Research Institute, Montreal, QC H2S 3H1, Canada

Unique Center, The Quebec Neuro-AI Research Center, Montreal, QC H3T 1P1, Canada

Institute Lyfe Research and Innovation Center, 69130 Écully, France

Funding

Prompt · BMU (Beam Me Up) · IVADO Excellence Scholarship · Union of AI and Neuroscience in Quebec (UNIQUE) · Canada Research Chairs program · a Discovery Grant · a New Investigators Award from the FRQNT · IVADO-Apogée fundamental research project grant

References (73)

Show source-provided references
1. Inducing Amusement in the Laboratory and Its Moderation by Experiential Approach10.1177/00332941211007935 ↗
2. Influence of gender on judgment of dark and nondark humor
3. Aslam, A., and Hussian, B. (2021). Emotion recognition techniques with rule based and machine learning approaches. arXiv.
4. Autonomic Response Patterns During Sadness and Mirth10.1111/j.1469-8986.1969.tb02840.x ↗
5. The effect of humor on short-term memory in older adults: A new component for whole-person wellness
6. Humors Effect on Short-term Memory in Healthy and Diabetic Older Adults
7. No Need to Laugh Out Loud: Predicting Humor Appraisal of Comic Strips Based on Physiological Signals in a Realistic Environment10.1145/3157730 ↗
8. Baveye, Y., Bettinelli, J.-N., Dellandréa, E., Chen, L., and Chamaret, C. (2013, January 2–5). A Large Video Database for Computational Models of Induced Emotion. Proceedings of the 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, Geneva, Switzerland.10.1109/ACII.2013.9 ↗
9. Bennett, M.P., Zeller, J.M., Rosenberg, L., and McCann, J. (2003). The Effect of Mirthful Laughter on Stress and Natural Killer Cell Activity, Western Kentucky University.
10. Bird, S., Klein, E., and Loper, E. (2009). Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit, O’Reilly Media, Inc.
11. Funny but aversive: A large-scale survey of the emotional response to Covid-19 humor in the Italian population during the lockdown10.1016/j.lingua.2020.102963 ↗
12. The openCV library
13. Random Forests10.1023/A:1010933404324 ↗
14. Sense of Humor, Stable Affect, and Psychological Well-Being10.5964/ejop.v10i3.746 ↗
15. Carretero-Dios, H., and Ruch, W. (2010). Humor Appreciation and Sensation Seeking: Invariance of Findings Across Culture and Assessment Instrument?. De Gruyter Brill.10.1515/humr.2010.020 ↗
16. Chandrasekaran, A., Vijayakumar, A.K., Antol, S., Bansal, M., Batra, D., Zitnick, C.L., and Parikh, D. (2016, January 27–30). We are humor beings: Understanding and predicting visual humor. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.498 ↗
17. The google similarity distance10.1109/TKDE.2007.48 ↗
18. Development and validation of brief measures of positive and negative affect: The PANAS scales10.1037/0022-3514.54.6.1063 ↗
19. Promoting emotional well-being through the use of humour10.1080/17439760.2011.577087 ↗
20. Dente, P., Küster, D., Skora, L., and Krumhuber, E. (2017, January 18–21). Measures and metrics for automatic emotion classification via FACET. Proceedings of the Conference on the Study of Artificial Intelligence and Simulation of Behaviour (AISB), Bath, UK.
21. Domínguez-Jiménez, J.A., Campo-Landines, K.C., Martínez-Santos, J.C., Delahoz, E.J., and Contreras-Ortiz, S.H. (2020). A machine learning model for emotion recognition from physiological signals. Biomed. Signal Process. Control, 55.10.1016/j.bspc.2019.101646 ↗
22. Are there basic emotions?10.1037/0033-295X.99.3.550 ↗
23. The Duchenne smile: Emotional expression and brain physiology: II10.1037/0022-3514.58.2.342 ↗
24. Ekman, P., and Friesen, W.V. (1978). Facial action coding system. Environ. Psychol. Nonverbal Behav.10.1037/t27734-000 ↗
25. Felt, false, and miserable smiles10.1007/BF00987191 ↗
26. Ekman, P., and Friesen, W.V. (2003). Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues, ISHK.
27. Relative Effectiveness of Emotion Induction Procedures and the Role of Personal Relevance in a Clinical Sample: A Comparison of Film, Images, and Music10.1007/s10862-011-9271-4 ↗
28. Frank, M.G., and Ekman, P. (1993). Not All Smiles Are Created Equal: The Differences Between Enjoyment and Nonenjoyment Smiles, De Gruyter Brill.10.1515/humr.1993.6.1.9 ↗
29. Behavioral markers and recognizability of the smile of enjoyment10.1037/0022-3514.64.1.83 ↗
30. Froehlich, E., Madipakkam, A.R., Craffonara, B., Bolte, C., Muth, A.-K., and Park, S.Q. (2021). A short humorous intervention protects against subsequent psychological stress and attenuates cortisol levels without affecting attention. Sci. Rep., 11.10.1038/s41598-021-86527-1 ↗
31. Differential sensitivity of humor ratings and mirth responses to cognitive and affective components of the humor response10.1037/0022-3514.51.1.209 ↗
32. How color properties can be used to elicit emotions in video games10.1155/2016/5182768 ↗
33. Ghai, M., Lal, S., Duggal, S., and Manik, S. (2017, January 23–25). Emotion recognition on speech signals using machine learning. Proceedings of the 2017 International Conference on Big Data Analytics and Computational Intelligence (ICBDAC), Chirala, India.10.1109/ICBDACI.2017.8070805 ↗
34. Geometric feature-based facial expression recognition in image sequences using multi-class adaboost and support vector machines10.3390/s130607714 ↗
35. Reconsidering the Duchenne Smile: Formalizing and Testing Hypotheses About Eye Constriction and Positive Emotion10.1007/s42761-020-00030-w ↗
36. Gunnery, S.D., and Hall, J.A. (2015). The expression and perception of the Duchenne smile. The Social Psychology of Nonverbal Communication, Springer Nature.10.1057/9781137345868_6 ↗
37. Herzog, T.R., and Karafa, J.A. (1998). Preferences for Sick Versus Nonsick Humor, De Gruyter Brill.10.1515/humr.1998.11.3.291 ↗
38. Gender differences in humor-related traits, humor appreciation, production, comprehension,(neural) responses, use, and correlates: A systematic review10.1007/s12144-020-00724-1 ↗
39. Hou, X., and Zhang, L. (2007, January 17–22). Saliency detection: A spectral residual approach. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.10.1109/CVPR.2007.383267 ↗
40. Jang, E.-H., Park, B.-J., Kim, S.-H., Chung, M.-A., Park, M.-S., and Sohn, J.-H. (2014, January 26–28). Emotion classification based on bio-signals emotion recognition using machine learning algorithms. Proceedings of the 2014 International Conference on Information Science, Electronics and Electrical Engineering, Sapporo, Japan.10.1109/InfoSEEE.2014.6946144 ↗
41. Quantifying dynamic facial expressions under naturalistic conditions10.7554/eLife.79581 ↗
42. Can Duchenne smiles be feigned? New evidence on felt and false smiles10.1037/a0017844 ↗
43. Kumar, P., Happy, S.L., and Routray, A. (2016, January 19–21). A real-time robust facial expression recognition system using HOG features. Proceedings of the 2016 International Conference on Computing, Analytics and Security Trends (CAST), Pune, India.10.1109/CAST.2016.7914982 ↗
44. Lefcourt, H.M., and Martin, R.A. (2012). Humor and Life Stress: Antidote to Adversity, Springer Science & Business Media.
45. Affective image classification by jointly using interpretable art features and semantic annotations10.1016/j.jvcir.2018.12.032 ↗
46. Martin, R.A. (2007). The social psychology of humor. The Psychology of Humor: An Integrative Approach, Routledge.10.1016/B978-012372564-6/50024-1 ↗
47. Martin, R.A., Kuiper, N.A., Olinger, L.J., and Dance, K.A. (1993). Humor, Coping with Stress, Self-Concept, and Psychological Well-Being, De Gruyter Brill.10.1515/humr.1993.6.1.89 ↗
48. Short-term mood change, perceived funniness, and the effect of humor stimuli10.1080/08964289.1996.9933763 ↗
49. PsychoPy2: Experiments in behavior made easy10.3758/s13428-018-01193-y ↗
50. On the use of computable features for film classification10.1109/TCSVT.2004.839993 ↗
51. Will the real relationship between facial expression and affective experience please stand up: The case of exhilaration10.1080/02699939508408964 ↗
52. State and Trait Cheerfulness and the Induction of Exhilaration10.1027/1016-9040.2.4.328 ↗
53. Ruch, W. (2001). The Perception of Humor, Word Scientific Publisher.10.1142/9789812810687_0032 ↗
54. Ruch, W., Heintz, S., Platt, T., Wagner, L., and Proyer, R.T. (2018). Broadening Humor: Comic Styles Differentially Tap into Temperament, Character, and Ability. Front. Psychol., 9.10.3389/fpsyg.2018.00006 ↗
55. Laughing to remember: Humor-related memory improvement
56. Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers10.1080/02699930903274322 ↗
57. Comparative study ID3, cart and C4. 5 decision tree algorithm: A survey
58. Facial expression analysis with AFFDEX and FACET: A validation study10.3758/s13428-017-0996-1 ↗
59. Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T., and Zeileis, A. (2008). Conditional variable importance for random forests. BMC Bioinform., 9.10.1186/1471-2105-9-307 ↗
60. Svebak, S. (1996). The Development of the Sense of Humor Questionnaire: From SHQ to SHQ-6, De Gruyter Brill.10.1515/humr.1996.9.3-4.341 ↗
61. Svebak, S., Martin, R.A., and Holmen, J. (2004). The Prevalence of Sense of Humor in a Large, Unselected County Population in Norway: Relations with Age, Sex, and Some Health Indicators, De Gruyter Brill.10.1515/humr.2004.001 ↗
62. Exploration of Individual Differences in Preferences for Humor10.2466/pr0.1974.34.3c.1031 ↗
63. Individual Differences in Reactions to Sexist Humor10.1177/1368430204039975 ↗
64. Recognizing Action Units for Facial Expression Analysis10.1109/34.908962 ↗
65. Toupin, G., Benlamine, M.S., and Frasson, C. (2021). Prediction of Amusement Intensity Based on Brain Activity. Novelties in Intelligent Digital Systems, IOS Press.10.3233/FAIA210099 ↗
66. Genetic and Environmental Contributions to Humor Styles: A Replication Study10.1375/twin.11.1.44 ↗
67. Spontaneous response to and expressive regulation of mirth elicited by humorous cartoons in younger and older adults10.1080/13825585.2018.1453922 ↗
68. Wang, B., and Dudek, P. (2014, January 23–28). A fast self-tuning background subtraction algorithm. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, OH, USA.10.1109/CVPRW.2014.64 ↗
69. Relative effectiveness and validity of mood induction procedures: A meta-analysis10.1002/(SICI)1099-0992(199607)26:4<557::AID-EJSP769>3.0.CO;2-4 ↗
70. Wu, Z., and Palmer, M. (1994). Verb semantics and lexical selection. arXiv.10.3115/981732.981751 ↗
71. A theory of humor elicitation10.1037/0033-295X.99.4.663 ↗
72. Effect of humour intervention programme on depression, anxiety, subjective well-being, cognitive function and sleep quality in Chinese nursing home residents10.1111/jan.14472 ↗
73. Zhao, S., Gao, Y., Jiang, X., Yao, H., Chua, T.-S., and Sun, X. (2014, January 3–7). Exploring Principles-of-Art Features For Image Emotion Recognition. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA.10.1145/2647868.2654930 ↗