Deep Learning Based Real Age and Gender Estimation from Unconstrained Face Image towards Smart Store Customer Relationship Management
Source abstract
Abstract
The COVID-19 pandemic markedly changed the human shopping nature, necessitating a contactless shopping system to curb the spread of the contagious disease efficiently. Consequently, a customer opts for a store where it is possible to avoid physical contacts and shorten the shopping process with extended services such as personalized product recommendations. Automatic age and gender estimation of a customer in a smart store strongly benefit the consumer by providing personalized advertisement and product recommendation; similarly, it aids the smart store proprietor to promote sales and develop an inventory perpetually for the future retail. In our paper, we propose a deep learning-founded enterprise solution for smart store customer relationship management (CRM), which allows us to predict the age and gender from a customer’s face image taken in an unconstrained environment to facilitate the smart store’s extended services, as it is expected for a modern venture. For the age estimation problem, we mitigate the data sparsity problem of the large public IMDB-WIKI dataset by image enhancement from another dataset and perform data augmentation as required. We handle our classification tasks utilizing an empirically leading pre-trained convolutional neural network (CNN), the VGG-16 network, and incorporate batch normalization. Especially, the age estimation task is posed as a deep classification problem followed by a multinomial logistic regression first-moment refinement. We validate our system for two standard benchmarks, one for each task, and demonstrate state-of-the-art performance for both real age and gender estimation.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Face recognition and analysis · Biometric Identification and Security · Face and Expression Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electronics and Information Engineering, Korea Aerospace University, Goyang 10540, Korea
Funding
Gyeonggi province, Korea
References (81)
Show source-provided references
1. Ricanek, K., and Tesafaye, T. (2006, January 10–12). Morph: A Longitudinal Image Database of Normal Adult Age-Progression. Proceedings of the Seventh International Conference on Automatic Face and Gesture Recognition, Southampton, UK.
2. Geng, X., Zhou, Z.-H., Zhang, Y., Li, G., and Dai, H. (2006, January 23–27). Learning from Facial Aging Patterns for Automatic Age Estimation. Proceedings of the ACM International Conference on Multimedia, Santa Barbara, CA, USA.10.1145/1180639.1180711 ↗
3. A lifespan database of adult facial stimuli10.3758/BF03206543 ↗
4. Age Estimation Using a Hierarchical Classifier Based on Global and Local Facial Features10.1016/j.patcog.2010.12.005 ↗
5. Song, Z., Ni, B., Guo, D., Sim, T., and Yan, S. (2011, January 6–13). Learning Universal Multi-View Age Estimator Using Video Context. Proceedings of the International Conference on Computer Vision, Barcelona, Spain.10.1109/ICCV.2011.6126248 ↗
6. Age synthesis and estimation via faces: A survey10.1109/TPAMI.2010.36 ↗
7. Overview of research on facial ageing using the FG-NET ageing database10.1049/iet-bmt.2014.0053 ↗
8. Face recognition and retrieval using cross-age reference coding with cross-age celebrity dataset10.1109/TMM.2015.2420374 ↗
9. Age and gender estimation of unfiltered faces10.1109/TIFS.2014.2359646 ↗
10. Han, H., Otto, C., and Jain, A.K. (2013, January 4–7). Age Estimation from Face Images: Human vs. Machine Performance. Proceedings of the 2013 International Conference on Biometrics (ICB), Madrid, Spain.10.1109/ICB.2013.6613022 ↗
11. Human Age Estimation and Sex Classification10.1007/978-3-642-28598-1_4 ↗
12. Asthana, A., Zafeiriou, S., Cheng, S., and Pantic, M. (2013, January 23–28). Robust Discriminative Response Map Fitting with Constrained Local Models. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA.10.1109/CVPR.2013.442 ↗
13. Localizing parts of faces using a consensus of exemplars10.1109/TPAMI.2013.23 ↗
14. Face alignment by explicit shape regression10.1007/s11263-013-0667-3 ↗
15. Dantone, M., Gall, J., Fanelli, G., and Van Gool, L. (2012, January 16–21). Real-Time Facial Feature Detection Using Conditional Regression Forests. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.10.1109/CVPR.2012.6247976 ↗
16. Deformable model fitting by regularized landmark mean-shift10.1007/s11263-010-0380-4 ↗
17. Xiong, X., and De la Torre, F. (2013, January 23–28). Supervised Descent Method and Its Applications to Face Alignment. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA.10.1109/CVPR.2013.75 ↗
18. Perceiving and remembering faces10.2307/1422222 ↗
19. Face recognition by humans: Nineteen results all computer vision researchers should know about10.1109/JPROC.2006.884093 ↗
20. Generic vs. person specific active appearance models10.1016/j.imavis.2005.07.009 ↗
21. Landmark-Guided Local Deep Neural Networks for Age and Gender Classification10.1155/2018/5034684 ↗
22. Rothe, R., Timofte, R., and Gool, L.V. (2016, January 27–30). Some Like It Hot-Visual Guidance for Preference Prediction. Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.599 ↗
23. Levi, G., and Hassner, T. (2015, January 7–12). Age and Gender Classification Using Convolutional Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPRW.2015.7301352 ↗
24. Deep expectation of real and apparent age from a single image without facial landmarks10.1007/s11263-016-0940-3 ↗
25. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA.
26. Human age estimation framework using different facial parts10.1016/j.eij.2011.02.002 ↗
27. Face recognition and age estimation implications of changes in facial features: A critical review study10.1109/ACCESS.2018.2836924 ↗
28. Young, H.K., and da Vitoria Lobo, N. (1994, January 21–23). Age Classification from Facial Images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.
29. Ramanathan, N., and Chellappa, R. (2006, January 17–22). Modeling Age Progression in Young Faces. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, NY, USA.
30. Image-based human age estimation by manifold learning and locally adjusted robust regression10.1109/TIP.2008.924280 ↗
31. Automatic Age Estimation Based on Facial Aging Patterns10.1109/TPAMI.2008.8 ↗
32. Yan, S., Liu, M., and Huang, T.S. (April, January 31). Extracting Age Information from Local Spatially Flexible Patches. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Las Vegas, NV, USA.
33. Yan, S., Zhou, X., Liu, M., Hasegawa-Johnson, M., and Huang, T.S. (2008, January 23–28). Regression from Patch-Kernel. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.
34. Fukunaga, K. (1990). Introduction to Statistical Pattern Recognition, Academic Press.10.1016/B978-0-08-047865-4.50007-7 ↗
35. An introduction to hidden markov models10.1109/MASSP.1986.1165342 ↗
36. Zhuang, X., Zhou, X., Hasegawa-Johnson, M., and Huang, T. (2008, January 8–11). Face Age Estimation Using Patch-Based Hidden Markov Model Supervectors. Proceedings of the International Conference on Pattern Recognition, Tampa, FL, USA.10.1109/ICPR.2008.4761364 ↗
37. Face Age Classification on Consumer Images with Gabor Feature and Fuzzy LDA Method10.1007/978-3-642-01793-3_14 ↗
38. Guo, G., Mu, G., Fu, Y., Dyer, C., and Huang, T. (October, January 29). A Study on Automatic Age Estimation Using a Large Database. Proceedings of the International Conference on Computer Vision, Kyoto, Japan.
39. Yang, Z., and Ai, H. (2007, January 27–29). Demographic Classification with Local Binary Patterns. Proceedings of the International Conference on Biometrics (ICB), Seoul, Korea.
40. Human age estimation with regression on discriminative aging manifold10.1109/TMM.2008.921847 ↗
41. Comparing different classifiers for automatic age estimation10.1109/TSMCB.2003.817091 ↗
42. Drucker, H., Burges, C.J.C., Kaufman, L., Smola, A.J., and Vapnik, V. (1996, January 2–5). Support Vector Regression Machines. Proceedings of the International Conference on Neural Information Processing Systems, Denver, CO, USA.
43. Partial least-squares regression: A tutorial10.1016/0003-2670(86)80028-9 ↗
44. Canonical correlation analysis: An overview with application to learning methods10.1162/0899766042321814 ↗
45. Support-vector networks10.1007/BF00994018 ↗
46. Demographic estimation from face images: Human vs. machine performance10.1109/TPAMI.2014.2362759 ↗
47. Zhang, Y., and Yeung, D.Y. (2010, January 13–18). Multi-Task Warped Gaussian Process for Personalized Age Estimation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5539975 ↗
48. Chen, K., Gong, S., Xiang, T., and Change Loy, C. (2013, January 23–28). Cumulative Attribute Space for Age and Crowd Density Estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.319 ↗
49. Chang, K.Y., Chen, C.S., and Hung, Y.P. (2011, January 20–25). Ordinal Hyperplanes Ranker with Cost Sensitivities for Age Estimation. Proceedings of the CVPR 2011, Colorado Springs, CO, USA.10.1109/CVPR.2011.5995437 ↗
50. A framework for joint estimation of age, gender and ethnicity on a large database10.1016/j.imavis.2014.04.011 ↗
51. Yi, D., Lei, Z., and Li, S.Z. (2014, January 1–5). Age Estimation by Multi-Scale Convolutional Network. Proceedings of the Asian Conference on Computer Vision (ACCV), Singapore.10.1007/978-3-319-16811-1_10 ↗
52. Wang, X., Guo, R., and Kambhamettu, C. (2015, January 5–9). Deeply-Learned Feature for Age Estimation. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV.2015.77 ↗
53. A hybrid deep learning CNN–ELM for age and gender classification10.1016/j.neucom.2017.08.062 ↗
54. Liu, X., Zou, Y., Kuang, H., and Ma, X. (2020). Face Image Age Estimation Based on Data Augmentation and Lightweight Convolutional Neural Network. Symmetry, 12.10.3390/sym12010146 ↗
55. Facial Age Estimation Using a Multi-Task Network Combining Classification and Regression
56. Enhanced Cycle Generative Adversarial Network for Generating Face Images of Untrained Races and Ages for Age Estimation10.1109/ACCESS.2020.3048369 ↗
57. Efficient Group-n Encoding and Decoding for Facial Age Estimation10.1109/TPAMI.2017.2779808 ↗
58. Niu, Z., Zhou, M., Wang, L., Gao, X., and Hua, G. (July, January 26). Ordinal Regression with Multiple Output CNN for Age Estimation. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.532 ↗
59. Evaluation of gender classification methods with automatically detected and aligned faces10.1109/TPAMI.2007.70800 ↗
60. Soft Biometrics for Surveillance: An overview
61. Golomb, B.A., Lawrence, D.T., and Sejnowski, T.J. (1990, January 26–29). Sexnet: A Neural Network Identifies Sex from Human Faces. Proceedings of the 1990 Conference on Advances in Neural Information Processing Systems, Denver, CO, USA.
62. Learning gender with support faces10.1109/34.1000244 ↗
63. Boosting sex identification performance10.1007/s11263-006-8910-9 ↗
64. Detection, localization, and sex classification of faces from arbitrary viewpoints and under occlusion10.1109/TPAMI.2008.233 ↗
65. A study on gait-based gender classification10.1109/TIP.2009.2020535 ↗
66. Semantic pyramids for gender and action recognition10.1109/TIP.2014.2331759 ↗
67. The Hidden Sides of Names—Face modeling with first name attributes10.1109/TPAMI.2014.2302443 ↗
68. Deeply Learned Classifiers for Age and Gender Predictions of Unfiltered Faces
69. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–6). Imagenet Classification with Deep Convolutional Neural Networks. Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS), Lake Tahoe, NV, USA.
70. Imagenet large scale visual recognition challenge10.1007/s11263-015-0816-y ↗
71. Girshick, R.B., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich Feature Hierarchies for Accurate Object Detection and Semantic Seg-Mentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
72. Object detection with discriminatively trained part-based models10.1109/TPAMI.2009.167 ↗
73. Mathias, M., Benenson, R., Pedersoli, M., and Gool, L.V. (2014, January 6–12). Face Detection without Bells and Whistles. Proceedings of the IEEE European Conference on Computer Vision (ECCV), Zurich, Switzerland.10.1007/978-3-319-10593-2_47 ↗
74. Joint face detection and alignment using multitask cascaded convolutional networks10.1109/LSP.2016.2603342 ↗
75. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2019, January 8–14). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Proceedings of the 33rd Conference on Neural Information Processing Systems (NIPS 2019), Vancouver, BC, Canada.
76. Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Li, F.-F. (2009, January 20–25). ImageNet: A Large-Scale Hierarchical Image Database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
77. Huerta, I., Fernández, C., and Prati, A. (2014, January 6–12). Facial Age Estimation through the Fusion of Texture and Local Appearance Descriptors. Proceedings of the IEEE European Conference on Computer Vision (ECCV), Zurich, Switzerland.10.1007/978-3-319-16181-5_51 ↗
78. Zhang, C., Liu, S., Xu, X., and Zhu, C. (2019, January 16–20). C3AE: Exploring the Limits of Compact Model for Age Estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.01287 ↗
79. Lapuschkin, S., Binder, A., Muller, K.-R., and Samek, W. (2017, January 22–29). Understanding and Comparing Deep Neural Networks for Age and Gender Classification. Proceedings of the IEEE International Conference on Computer Vision Workshop, Venice, Italy.
80. Hassner, T., Harel, S., Paz, E., and Enbar, R. (2015, January 7–12). Effective Face Frontalization in Unconstrained Images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7299058 ↗
81. Khan, K., Attique, M., Syed, I., Sarwar, G., Irfan, M.A., and Khan, R.U. (2019). A Unified Framework for Head Pose, Age and Gender Classification through End-to-End Face Segmentation. Entropy, 21.10.3390/e21070647 ↗