Multimodal Biometric Template Protection Based on a Cancelable SoftmaxOut Fusion Network
Source abstract
Abstract
Authentication systems that employ biometrics are commonplace, as they offer a convenient means of authenticating an individual’s identity. However, these systems give rise to concerns about security and privacy due to insecure template management. As a remedy, biometric template protection (BTP) has been developed. Cancelable biometrics is a non-invertible form of BTP in which the templates are changeable. This paper proposes a deep-learning-based end-to-end multimodal cancelable biometrics scheme called cancelable SoftmaxOut fusion network (CSMoFN). By end-to-end, we mean a model that receives raw biometric data as input and produces a protected template as output. CSMoFN combines two biometric traits, the face and the periocular region, and is composed of three modules: a feature extraction and fusion module, a permutation SoftmaxOut transformation module, and a multiplication-diagonal compression module. The first module carries out feature extraction and fusion, while the second and third are responsible for the hashing of fused features and compression. In addition, our network is equipped with dual template-changeability mechanisms with user-specific seeded permutation and binary random projection. CSMoFN is trained by minimizing the ArcFace loss and the pairwise angular loss. We evaluate the network, using six face–periocular multimodal datasets, in terms of its verification performance, unlinkability, revocability, and non-invertibility.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Biometric Identification and Security · Face recognition and analysis · Advanced Steganography and Watermarking Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electrical and Electronic Engineering, College of Engineering, Yonsei University, Seoul 03722, Korea
Department of Electrical and Computer Engineering, Northeastern University, Boston, MA 02115, USA
References (43)
Show source-provided references
1. Biometric Template Security10.1155/2008/579416 ↗
2. An Introduction to Biometric Recognition10.1109/TCSVT.2003.818349 ↗
3. Generating Cancelable Fingerprint Templates10.1109/TPAMI.2007.1004 ↗
4. Jain, A.K., Ross, A.A., and Nandakumar, K. (2011). Introduction to Biometrics, Springer Science & Business Media.10.1007/978-0-387-77326-1 ↗
5. Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects10.1109/JPROC.2015.2460697 ↗
6. Unimodal and Multimodal Biometric Sensing Systems: A Review10.1109/ACCESS.2016.2614720 ↗
7. Investigating Fusion Approaches in Multi-Biometric Cancellable Recognition10.1016/j.eswa.2012.10.002 ↗
8. Pinto, J.R., Cardoso, J.S., and Correia, M.V. (2020, January 29–30). Secure Triplet Loss for End-to-End Deep Biometrics. Proceedings of the 2020 8th International Workshop on Biometrics and Forensics (IWBF), Porto, Portugal.10.1109/IWBF49977.2020.9107958 ↗
9. Robust Face Recognition Via Multimodal Deep Face Representation10.1109/TMM.2015.2477042 ↗
10. A multi-biometric iris recognition system based on a deep learning approach10.1007/s10044-017-0656-1 ↗
11. Alay, N., and Al-Baity, H.H. (2020). Deep Learning Approach for Multimodal Biometric Recognition System Based on Fusion of Iris, Face, and Finger Vein Traits. Sensors, 20.10.3390/s20195523 ↗
12. Deep Multimodal Biometric Recognition Using Contourlet Derivative Weighted Rank Fusion with Human Face, Fingerprint and Iris Images
13. Implementation of Multimodal Biometric Recognition Via Multi-Feature Deep Learning Networks and Feature Fusion10.1007/s11042-019-7618-0 ↗
14. Hierarchical Fusion Network for Periocular and Iris by Neural Network Approximation and Sparse Autoencoder10.1007/s00138-020-01140-y ↗
15. A Deep Feature Fusion Network Based on Multiple Attention Mechanisms for Joint Iris-Periocular Biometric Recognition10.1109/LSP.2021.3079850 ↗
16. Periocular Recognition in the Wild With Generalized Label Smoothing Regularization10.1109/LSP.2020.3014472 ↗
17. Soleymani, S., Torfi, A., Dawson, J., and Nasrabadi, N.M. (2018, January 7–10). Generalized Bilinear Deep Convolutional Neural Networks for Multimodal Biometric Identification. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.10.1109/ICIP.2018.8451532 ↗
18. Multi-Biometric Template Protection Based on Bloom Filters10.1016/j.inffus.2017.10.003 ↗
19. Two Feature-Level Fusion Methods with Feature Scaling and Hashing For Multimodal Biometrics10.1080/02564602.2016.1149039 ↗
20. A Fingerprint and Finger-Vein Based Cancelable Multi-Biometric System10.1016/j.patcog.2018.01.026 ↗
21. A Tokenless Cancellable Scheme for Multimodal Biometric Systems10.1016/j.cose.2021.102350 ↗
22. Novel Approach for Multimodal Feature Fusion to Generate Cancelable Biometric10.1007/s00371-020-01873-x ↗
23. Cancelable Multi-Biometric Recognition System Based on Deep Learning10.1007/s00371-019-01715-5 ↗
24. Deep Hashing for Secure Multimodal Biometrics10.1109/TIFS.2020.3033189 ↗
25. Deep Learning for Multi-Instance Biometric Privacy
26. El-Rahiem, B.A., Amin, M., Sedik, A., Samie, F.E., and Iliyasu, A.M. An efficient multi-biometric cancellable biometric scheme based on deep fusion and deep dream, J. Ambient. Intell. Humaniz. Comput., in press.
27. Random Permutation Maxout Transform for Cancellable Facial Template Protection10.1007/s11042-018-5956-y ↗
28. A Survey on Learning to Hash10.1109/TPAMI.2017.2699960 ↗
29. Du, H., Shi, H., Zeng, D., and Mei, T. (2009). The Elements of End-to-End Deep Face Recognition: A Survey of Recent Advances. arXiv.
30. Leibe, B., Matas, J., and Sebe, N. (2016, January 11–14). Ms-celeb-1m: A Dataset and Benchmark for Large-Scale Face Recognition. Proceedings of the European Conference on Computer Vision, Amstedrdam, The Netherlands.
31. Lee, H., Low, C.Y., and Teoh, A.B.J. (2021, January 10–15). SoftmaxOut Transformation-Permutation Network for Facial Template Protection. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.10.1109/ICPR48806.2021.9413163 ↗
32. Li, W., and Zhang, S. (2020). Binary Random Projections with Controllable Sparsity Patterns. arXiv.
33. Jin, A.T.B. (2006, January 17–22). Cancellable biometrics and multispace random projections. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop (CVPRW’06), New York, NY, USA.
34. Deng, J., Guo, J., Xue, N., and Zafeiriou, S. (2019, January 16–20). Arcface: Additive Angular Margin Loss for Deep Face Recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00482 ↗
35. Martinez, A., and Benavente, R. (1998). The AR Face Database: CVC Technical Report, 24, Centre de Visioper Computador Universitat Aut onoma de Barcelona.
36. Tiong, L.C.O., Teoh, A.B.J., and Lee, Y. (2019, January 4–7). Periocular Recognition in the Wild with Orthogonal Combination of Local Binary Coded Pattern in Dual-Stream Convolutional Neural Network. Proceedings of the 2019 International Conference on Biometrics (ICB), Crete, Greece.
37. Ng, H.-W., and Winkler, S. (2014, January 27–30). A Data-Driven Approach to Cleaning Large Face Datasets. Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), Paris, France.10.1109/ICIP.2014.7025068 ↗
38. Rothe, R., Timofte, R., and Van Gool, L. (2015, January 7–13). Dex: Deep Expectation of Apparent Age from a Single Image. Proceedings of the IEEE International Conference on Computer Vision Workshops, Santiago, Chile.10.1109/ICCVW.2015.41 ↗
39. Kumar, N., Berg, A.C., Belhumeur, P.N., and Nayar, S.K. (October, January 29). Attribute and simile classifiers for face verification. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.10.1109/ICCV.2009.5459250 ↗
40. Wolf, L., Hassner, T., and Maoz, I. (2011, January 20–25). Face Recognition in Unconstrained Videos with Matched Background Similarity. Proceedings of the CVPR 2011, Colorado Springs, CO, USA.10.1109/CVPR.2011.5995566 ↗
41. General Framework to Evaluate Unlinkability in Biometric Template Protection Systems
42. Ranking-Based Locality Sensitive Hashing-Enabled Cancelable Biometrics: Index-of-Max Hashing10.1109/TIFS.2017.2753172 ↗
43. Security Considerations in Minutiae-Based Fuzzy Vaults10.1109/TIFS.2015.2392559 ↗