A Semi-Lightweight Multi-Feature Integration Architecture for Micro-Expression Recognition
Source abstract
Abstract
Micro-expressions, fleeting involuntary facial cues lasting under half a second, reveal genuine emotions and are valuable in clinical diagnosis and psychotherapy. Real-time recognition on resource-constrained embedded devices remains challenging, as current methods struggle to balance performance and efficiency. This study introduces a semi-lightweight multifunctional network that enhances real-time deployment and accuracy. Unlike prior simplistic feature fusion techniques, our novel multi-feature fusion strategy leverages temporal, spatial, and differential features to better capture dynamic changes. Enhanced by Residual Network (ResNet) architecture with channel and spatial attention mechanisms, the model improves feature representation while maintaining a lightweight design. Evaluations on SMIC, CASME II, SAMM, and their composite dataset show superior performance in Unweighted F1 Score (UF1) and Unweighted Average Recall (UAR), alongside faster detection speeds compared to existing algorithms.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Web Data Mining and Analysis · Machine Learning and Data Classification · Video Analysis and Summarization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (47)
Show source-provided references
1. Reading between the lies: identifying concealed and falsified emotions in universal facial expressions10.1111/j.1467-9280.2008.02116.x ↗
2. Police lie detection accuracy: the effect of lie scenario10.1007/s10979-008-9166-4 ↗
3. Abnormal recognition of facial expression of emotions in depressed patients with major depression disorder and schizotypal personality disorder10.1016/0006-3223(96)00032-7 ↗
4. I see how you feel: training laypeople and professionals to recognize fleeting emotions
5. A comparative study of texture measures with classification based on featured distributions10.1016/0031-3203(95)00067-4 ↗
6. Determining optical flow10.1016/0004-3702(81)90024-2 ↗
7. Support-vector networks10.1023/A:1022627411411 ↗
8. Random forests10.1023/A:1010933404324 ↗
9. Selective deep features for micro-expression recognition
10. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An image is worth 16 × 16 words: transformers for image recognition at scale. arXiv:2010.11929. 2020.
11. Deep residual learning for image recognition
12. Recognising spontaneous facial micro-expressions
13. LBP with six intersection points: reducing redundant information in LBP-TOP for micro-expression recognition
14. Facial micro-expression recognition using spatiotemporal local binary pattern with integral projection
15. Efficient spatiotemporal local binary patterns for spontaneous facial micro-expression recognition10.1371/journal.pone.0124674 ↗
16. Discriminative spatiotemporal local binary pattern with revisited integral projection for spontaneous facial micro-expression recognition10.1109/TAFFC.2017.2713359 ↗
17. A main directional mean optical flow feature for spontaneous micro-expression recognition10.1109/TAFFC.2015.2485205 ↗
18. Microexpression identification and categorization using a facial dynamics map10.1109/TAFFC.2016.2518162 ↗
19. Micro-expression recognition by aggregating local spatio-temporal patterns
20. Less is more: micro-expression recognition from video using apex frame10.1016/j.image.2017.11.006 ↗
21. Fuzzy histogram of optical flow orientations for micro-expression recognition10.1109/TAFFC.2017.2723386 ↗
22. Off-apexnet on micro-expression recognition system10.1016/j.image.2019.02.005 ↗
23. Capsulenet for micro-expression recognition
24. Shallow triple stream three-dimensional CNN (STSTNet) for micro-expression recognition
25. Revealing the invisible with model and data shrinking for composite-database micro-expression recognition10.1109/TIP.2020.3018222 ↗
26. Feature refinement: an expression-specific feature learning and fusion method for micro-expression recognition10.1016/j.patcog.2021.108275 ↗
27. Micro-expression recognition based on facial graph representation learning and facial action unit fusion
28. ME-PLAN: a deep prototypical learning with local attention network for dynamic micro-expression recognition10.1016/j.neunet.2022.06.024 ↗
29. RNAS-MER: a refined neural architecture search with hybrid spatiotemporal operations for micro-expression recognition
30. Geometric graph representation with learnable graph structure and adaptive au constraint for micro-expression recognition10.1109/TAFFC.2023.3340016 ↗
31. Multimodal fusion-based Swin transformer for facial recognition micro-expression recognition
32. Feature representation learning with adaptive displacement generation and transformer fusion for micro-expression recognition
33. HTNet for micro-expression recognition10.1016/j.neucom.2024.128196 ↗
34. JGULF: joint global and unilateral local feature network for micro-expression recognition10.1016/j.imavis.2024.105091 ↗
35. Boosting micro-expression recognition via self-expression reconstruction and memory contrastive learning10.1109/TAFFC.2024.3397701 ↗
36. HFA-Net: hierarchical feature aggregation network for micro-expression recognition10.1007/s40747-024-01660-4 ↗
37. Temporal augmented contrastive learning for micro-expression recognition10.1016/j.patrec.2023.02.003 ↗
38. Wang J, Tian Y, Yang Y, Chen X, Zheng C, Qiang W. Meta-auxiliary learning for micro-expression recognition. arXiv:2024.12024. 2024.
39. A spontaneous micro-expression database: inducement, collection and baseline
40. Automatic apex frame spotting in micro-expression database
41. Face recognition with local binary patterns
42. CASME II: an improved spontaneous micro-expression database and the baseline evaluation10.1371/journal.pone.0086041 ↗
43. SAMM: a spontaneous micro-facial movement dataset10.1109/TAFFC.2016.2573832 ↗
44. SAMM long videos: a spontaneous facial micro-and macro-expressions dataset
45. MEGC 2019—the second facial micro-expressions grand challenge
46. Prechelt · Neural networks: tricks of the trade · 2002 · 55
47. Gradient descent finds global minima of deep neural networks