Text- and Face-Conditioned Multi-Anchor Conditional Embedding for Robust Periocular Recognition
Source abstract
Abstract
Periocular recognition is essential when full-face images cannot be used because of occlusion, privacy constraints, or sensor limitations, yet in many deployments, only periocular images are available at run time, while richer evidence, such as archival face photos and textual metadata, exists offline. This mismatch makes it hard to deploy conventional multimodal fusion. This motivates the notion of conditional biometrics, where auxiliary modalities are used only during training to learn stronger periocular representations while keeping deployment strictly periocular-only. In this paper, we propose Multi-Anchor Conditional Periocular Embedding (MACPE), which maps periocular, facial, and textual features into a shared anchor-conditioned space via a learnable anchor bank that preserves periocular micro-textures while aligning higher-level semantics. Training combines identity classification losses on periocular and face branches with a symmetric InfoNCE loss over anchors and a pulling regularizer that jointly aligns periocular, facial, and textual embeddings without collapsing into face-dominated solutions; captions generated by a vision language model provide complementary semantic supervision. At deployment, only the periocular encoder is used. Experiments across five periocular datasets show that MACPE consistently improves Rank-1 identification and reduces EER at a fixed FAR compared with periocular-only baselines and alternative conditioning methods. Ablation studies verify the contributions of anchor-conditioned embeddings, textual supervision, and the proposed loss design.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Biometric Identification and Security · Face recognition and analysis · Reconstructive Facial Surgery Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Republic of Korea
References (43)
Show source-provided references
1. Kortli, Y., Jridi, M., Al Falou, A., and Atri, M. (2020). Face Recognition Systems: A Survey. Sensors, 20.10.3390/s20020342 ↗
2. A survey of face recognition techniques under occlusion10.1049/bme2.12029 ↗
3. Jadhav, S.B., Deshmukh, N.K., Lomate, S.M., Pawar, S.B., and Hambarde, K.A. (2024, January 16–17). Integrated multimodal Face, Iris and Fingerprint biometric fusion for advanced person identification using Hybrid Deep Learning. Proceedings of the National Conference on Machine Learning and Data Science (NCMLDS-2024), Latur, India.
4. Ammour, B., Boubchir, L., Bouden, T., and Ramdani, M. (2020). Face-Iris Multimodal Biometric Identification System. Electronics, 9.10.3390/electronics9010085 ↗
5. Periocular Biometrics: A Modality for Unconstrained Scenarios
6. Alonso-Fernandez, F., and Bigun, J. (2016, January 3–4). Periocular biometrics: Databases, algorithms and directions. Proceedings of the 2016 4th International Conference on Biometrics and Forensics (IWBF), Limassol, Cyprus.10.1109/IWBF.2016.7449688 ↗
7. A survey on periocular biometrics research10.1016/j.patrec.2015.08.026 ↗
8. Periocular biometrics and its relevance to partially masked faces: A survey10.1016/j.cviu.2022.103583 ↗
9. A new periocular dataset collected by mobile devices in unconstrained scenarios10.1038/s41598-022-22811-y ↗
10. Convolutional Tri Biometrics: A Unified Approach for Iris, Periocular, and Facial Authentication
11. Person Identification Using Fusion of Iris and Periocular Deep Features10.1016/j.neunet.2019.11.009 ↗
12. On the Representation Learning of Conditional Biometrics for Flexible Deployment10.1109/ACCESS.2023.3301150 ↗
13. Attribute-based deep periocular recognition: Leveraging soft biometrics to improve periocular recognition
14. Jin, C. (2023). Semi-Supervised Image Captioning with CLIP. arXiv.
15. Limbu, M., and Banerjee, D. (2025). MedBLIP: Fine-tuning BLIP for Medical Image Captioning. arXiv.
16. Yildirim, T., Amasyali, M., and Karaca, A.C. (2025, January 25–28). LLaVa-RS: A Unified Model for Image and Change Captioning in Remote Sensing. Proceedings of the 2025 30th Signal Processing and Communications Applications Conference (SIU), Şile, Turkey.10.1109/SIU66497.2025.11112506 ↗
17. Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., and Zhou, J. (2023). Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond. arXiv.
18. LBP-based periocular recognition on challenging face datasets10.1186/1687-5281-2013-36 ↗
19. Santos, G., and Proença, H. (2013, January 16–19). Periocular biometrics: An emerging technology for unconstrained scenarios. Proceedings of the 2013 IEEE Symposium on Computational Intelligence in Biometrics and Identity Management (CIBIM), Singapore.10.1109/CIBIM.2013.6607908 ↗
20. Padole, C.N., and Proenca, H. (April, January 29). Periocular recognition: Analysis of performance degradation factors. Proceedings of the 2012 5th IAPR International Conference on Biometrics (ICB), New Delhi, India.10.1109/ICB.2012.6199790 ↗
21. Ng, T.S., Low, C.Y., Long Chai, J.C., and Beng Jin Teoh, A. (2022, January 21–25). Conditional Multimodal Biometrics Embedding Learning For Periocular and Face in the Wild. Proceedings of the 2022 26th International Conference on Pattern Recognition (ICPR), Montreal, QC, Canada.10.1109/ICPR56361.2022.9956636 ↗
22. Ow Tiong, L.C., Sigmund, D., Chan, C.H., and Jin Teoh, A.B. (2024, January 16–22). Flexible Biometrics Recognition: Bridging the Multimodality Gap Through Attention, Alignment and Prompt Tuning. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.00033 ↗
23. Self-Attentive Contrastive Learning for Conditioned Periocular and Face Biometrics10.1109/TIFS.2024.3361216 ↗
24. Hasan, M.M., Sami, S.M., and Nasrabadi, N.M. (2023, January 2–7). Improving Face Recognition from Caption Supervision with Multi-Granular Contextual Feature Aggregation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/IJCB57857.2023.10448749 ↗
25. Hasan, M.M., Sami, S.M., and Nasrabadi, N.M. (2024, January 3–8). Text-Guided Face Recognition using Multi-Granularity Cross-Modal Contrastive Learning. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACV57701.2024.00568 ↗
26. Learning Multi-Scale Knowledge-Guided Features for Text-Guided Face Recognition10.1109/TBIOM.2024.3466216 ↗
27. Parkhi, O.M., Vedaldi, A., and Zisserman, A. (2015, January 7–10). Deep Face Recognition. Proceedings of the British Machine Vision Conference (BMVC), Swansea, UK.
28. Multimodal facial biometrics recognition: Dual-stream convolutional neural networks with multi-feature fusion layers10.1016/j.imavis.2020.103977 ↗
29. Kumar, N., Berg, A.C., Belhumeur, P.N., and Nayar, S.K. (October, January 29). Attribute and simile classifiers for face verification. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.10.1109/ICCV.2009.5459250 ↗
30. Ng, H.W., and Winkler, S. (2014, January 27–30). A data-driven approach to cleaning large face datasets. Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), Paris, France.10.1109/ICIP.2014.7025068 ↗
31. Rothe, R., Timofte, R., and Van Gool, L. (2015, January 7–13). DEX: Deep EXpectation of Apparent Age from a Single Image. Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), Santiago, Chile.10.1109/ICCVW.2015.41 ↗
32. Martinez, A., and Benavente, R. (1998). The AR Face Database, Computer Vision Center (CVC), Universitat Autònoma de Barcelona (UAB). Technical Report 24.
33. A survey on multimodal large language models10.1093/nsr/nwae403 ↗
34. Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., and Ge, W. (2024). Qwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution. arXiv.
35. Li, J., Li, D., Xiong, C., and Hoi, S. (2022, January 17–23). BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. Proceedings of the ICML, Baltimore, MD, USA.
36. Li, J., Li, D., Savarese, S., and Hoi, S. (2023, January 23–29). BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. Proceedings of the 40th International Conference on Machine Learning, JMLR.org, ICML’23, Honolulu, HI, USA.
37. Zhu, Y., Xu, H., Du, A., and Wang, B. (2024). Image–Text Matching Model Based on CLIP Bimodal Encoding. Appl. Sci., 14.10.3390/app142210384 ↗
38. Attention-aware ensemble learning for face-periocular cross-modality matching10.1016/j.asoc.2025.113044 ↗
39. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv.
40. Chen, S., Liu, Y., Gao, X., and Han, Z. (2018, January 11–12). MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices. Proceedings of the Biometric Recognition, Urumchi, China.10.1007/978-3-319-97909-0_46 ↗
41. Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., and Liu, W. (2018, January 18–23). CosFace: Large Margin Cosine Loss for Deep Face Recognition. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00552 ↗
42. Cui, J., Zhang, Y.G., Liu, H., and Wang, Y. (2025). An Inclusive Theoretical Framework of Robust Supervised Contrastive Loss against Label Noise. arXiv.
43. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗