Occlusion-Robust Facial Expression Recognition Based on Multi-Angle Feature Extraction

Facial occlusion represents a significant challenge in the domain of facial expression recognition (FER). The absence of feature information due to occlusion has been demonstrated to result in a reduction in recognition accuracy and model robustness. To address this challenge, a multi-angle feature extraction (MAFE) method is proposed in this paper, aiming to enhance the recognition accuracy under occlusion conditions by employing multi-scale global features, local fine-grained features, and important regional features. The MAFE approach involves three core modules: multi-feature extraction, regional detail feature fusion, and consistent feature recognition. In the multi-feature extraction module, PTIR-50 and Swin Transformer are used to extract global features and fine-grained features, and at the same time, the five key points of the face are combined to crop out the important regions from the global features. The Regional Bias Loss (RB-Loss) is then utilized to guide the model to focus on the key information regions. The subsequent Regional Detail Feature Fusion module combines fine-grained features with those from the important regions. This process enhances the expressiveness of the features. The Consistent Feature Recognition module proposes consistent feature loss (con-feature Loss) to ensure that global features and fused features guide each other, forcing the model to focus on more discriminative expression features. The experimental results demonstrate that MAFE attains 89.42% and 86.94% accuracies on the Occlusion-RAFDB and Occlusion-FERPlus datasets, thereby surpassing the existing methods. Accuracies of 92.11% and 90.15% are also obtained on the original RAF-DB and FERPlus datasets.