FP-Deeplab: A Novel Face Parsing Network for Fine-Grained Boundary Detection and Semantic Understanding
Source abstract
Abstract
Facial semantic segmentation, as a critical technology in high-level visual understanding, plays an important role in applications such as facial editing, augmented reality, and identity recognition. However, due to the complexity of facial structures, ambiguous boundaries, and inconsistent scales of facial components, traditional methods still suffer from significant limitations in detail preservation and contextual modeling. To address these challenges, this paper proposes a facial parsing network based on the Deeplabv3+ framework, named FP-Deeplab, which aims to improve segmentation performance and generalization capability through structurally enhanced modules. Specifically, two key modules are designed: (1) the Context-Channel Refine Feature Enhancement (CCR-FE) module, which integrates multi-scale contextual strip convolutions and Cross-Axis Attention and introduces a channel attention mechanism to strengthen the modeling of long-range spatial dependencies and enhances the perception and representation of boundary regions; (2) the Self-Modulation Attention Feature Integration with Regularization (SimFA) module, which combines local detail modeling and a parameter-free channel attention modulation mechanism to achieve fine-grained reconstruction and enhancement of semantic features, effectively mitigating boundary blur and information loss during the upsampling stage. The experimental results on two public facial segmentation datasets, CelebAMask-HQ and HELEN, demonstrate that FP-Deeplab improves the baseline model by 3.8% in Mean IoU and 2.3% in the overall F1-score on the HELEN dataset, and it achieves a Mean F1-score of 84.8% on the CelebAMask-HQ dataset. Furthermore, the proposed method shows superior accuracy and robustness in multiple key component categories, especially in long-tailed regions, validating its effectiveness.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Face recognition and analysis · Advanced Neural Network Applications · Advanced Image and Video Retrieval Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Information Engineering, Sichuan Agricultural University, Ya’an 625000, China
College of Art and Media, Sichuan Agricultural University, Ya’an 625000, China
Sichuan Key Laboratory of Agricultural Information Engineering, Ya’an 625014, China
References (43)
Show source-provided references
1. Anthes, C., García-Hernández, R.J., Wiedemann, M., and Kranzlmüller, D. (2016, January 5–12). State of the art of virtual reality technology. Proceedings of the 2016 IEEE Aerospace Conference, Big Sky, MT, USA.10.1109/AERO.2016.7500674 ↗
2. Arena, F., Collotta, M., Pau, G., and Termine, F. (2022). An overview of augmented reality. Computers, 11.10.3390/computers11020028 ↗
3. A novel image expression-driven modeling strategy for coke quality prediction in the smart cokemaking process10.1016/j.energy.2024.130866 ↗
4. A review of semantic segmentation using deep neural networks10.1007/s13735-017-0141-z ↗
5. Computer vision and image processing: A paper review10.29099/ijair.v2i1.42 ↗
6. Convolutional neural networks: An overview and application in radiology10.1007/s13244-018-0639-9 ↗
7. Attention is all you need
8. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8–14). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_49 ↗
9. Yuan, H., and Ji, S. (2020, January 26–30). StructPool: Structured graph pooling via conditional random fields. Proceedings of the 8th International Conference on Learning Representations (ICLR), Addis Ababa, Ethiopia.
10. Deep learning Markov random field for semantic segmentation10.1109/TPAMI.2017.2737535 ↗
11. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
12. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015, Proceedings of the 18th International Conference, Munich, Germany, 5–9 October 2015, Springer. Proceedings, Part III 18.10.1007/978-3-319-24574-4_28 ↗
13. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs10.1109/TPAMI.2017.2699184 ↗
14. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
15. Yuan, Y., Chen, X., and Wang, J. (2020). Object-contextual representations for semantic segmentation. Computer Vision–ECCV 2020, Proceedings of the 16th European Conference, Glasgow, UK, 23–28 August 2020, Springer. Proceedings, Part VI 16.10.1007/978-3-030-58539-6_11 ↗
16. SegFormer: Simple and efficient design for semantic segmentation with transformers
17. Zhou, Y., Hu, X., and Zhang, B. (2015). Interlinked convolutional neural networks for face parsing. Advances in Neural Networks–ISNN 2015, Proceedings of the 12th International Symposium on Neural Networks, ISNN 2015, Jeju, South Korea, 15–18 October 2015, Springer.10.1007/978-3-319-25393-0_25 ↗
18. Luo, L., Xue, D., and Feng, X. (2020). Ehanet: An effective hierarchical aggregation network for face parsing. Appl. Sci., 10.10.3390/app10093135 ↗
19. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
20. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00986 ↗
21. Yang, L., Song, Q., Wang, Z., and Jiang, M. (2019, January 15–20). Parsing R-CNN for instance-level human analysis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00045 ↗
22. Yuan, Y., Fu, R., Huang, L., Lin, W., Zhang, C., Chen, X., and Wang, J. (2021). Hrformer: High-resolution transformer for dense prediction. arXiv.
23. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
24. Ho, J., Kalchbrenner, N., Weissenborn, D., and Salimans, T. (2019). Axial attention in multidimensional transformers. arXiv.
25. Shao, H., Zeng, Q., Hou, Q., and Yang, J. (2023). Mcanet: Medical image segmentation with multi-scale cross-axis attention. arXiv.
26. Agarap, A.F. (2018). Deep learning using rectified linear units (ReLU). arXiv.
27. Zheng, M., Sun, L., Dong, J., and Pan, J. (2024). SMFANet: A lightweight self-modulation feature aggregation network for efficient image super-resolution. Computer Vision–ECCV 2024, Proceedings of the European Conference on Computer Vision, Milan, Italy, 29 September–4 October 2024, Springer.10.1007/978-3-031-72973-7_21 ↗
28. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
29. Hendrycks, D., and Gimpel, K. (2016). Gaussian error linear units (gelus). arXiv.
30. Qin, X., Li, N., Weng, C., Su, D., and Li, M. (2022, January 22–27). Simple attention module based speaker verification with iterative noisy label detection. Proceedings of the ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9746294 ↗
31. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning10.1016/j.neunet.2017.12.012 ↗
32. Smith, B.M., Zhang, L., Brandt, J., Lin, Z., and Yang, J. (2013, January 23–28). Exemplar-based face parsing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.447 ↗
33. Lee, C.-H., Liu, Z., Wu, L., and Luo, P. (2020, January 13–19). MaskGAN: Towards diverse and interactive facial image manipulation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00559 ↗
34. Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., and Savarese, S. (2019, January 15–20). Generalized intersection over union: A metric and a loss for bounding box regression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00075 ↗
35. Opitz, J., and Burst, S. (2019). Macro f1 and macro f1. arXiv.
36. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. (2018, January 18–23). MobileNetV2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
37. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
38. End-to-end face parsing via interlinked convolutional neural networks10.1007/s11571-020-09615-4 ↗
39. Accurate facial image parsing at real-time speed10.1109/TIP.2019.2909652 ↗
40. Liu, S., Yang, J., Huang, C., and Yang, M.-H. (2015, January 7–12). Multi-objective convolutional learning for face labeling. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.
41. Liu, S., Shi, J., Liang, J., and Yang, M.-H. (2017). Face parsing via recurrent propagation. arXiv.10.5244/C.31.8 ↗
42. Guo, T., Kim, Y., Zhang, H., Qian, D., Yoo, B., Xu, J., Zou, D., Han, J.-J., and Choi, C. (2018, January 2–7). Residual encoder decoder network and adaptive prior for face parsing. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12268 ↗
43. Zheng, Y., Yang, H., Zhang, T., Bao, J., Chen, D., Huang, Y., Yuan, L., Chen, D., Zeng, M., and Wen, F. (2022, January 18–24). General facial representation learning in a visual-linguistic manner. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01814 ↗