Face Mask Detection on Photo and Real-Time Video Images Using Caffe-MobileNetV2 Transfer Learning
Source abstract
Abstract
Face detection systems have generally been used primarily for non-masked faces, which include relevant facial characteristics such as the ears, chin, lips, nose, and eyes. Masks are necessary to cover faces in many situations, such as pandemics, crime scenes, medical settings, high pollution, and laboratories. The COVID-19 epidemic has increased the requirement for people to use protective face masks in public places. Analysis of face detection technology is crucial with blocked faces, which typically have visibility only in the periocular area and above. This paper aims to implement a model on complex data, i.e., by taking tasks for the face detection of people from the photo and in real-time video images with and without a mask. This task is implemented based on the features around their eyes, ears, nose, and forehead by using the original masked and unmasked images to form a baseline for face detection. The idea of performing such a task is by using the Caffe-MobileNetV2 (CMNV2) model for feature extraction and masked image classification. The convolutional architecture for the fast feature embedding Caffe model is used as a face detector, and the MobileNetV2 is used for mask identification. In this work, five different layers are added to the pre-trained MobileNetV2 architecture for better classification accuracy with fewer training parameters for the given data for face mask detection. Experimental results revealed that the proposed methodology performed well, with an accuracy of 99.64% on photo images and good accuracy on real-time video images. Other metrics show that the model outperforms previous models with a precision of 100%, recall of 99.28%, f1-score of 99.64%, and an error rate of 0.36%. Face mask detection was originally a form of computing application, but it is now widely used in other technological areas such as smartphones and artificial intelligence. Computer-based masked-face detection belongs in the category of biometrics, since it includes using a person’s unique features to identify them with a mask on.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Face recognition and analysis · Face and Expression Recognition · Biometric Identification and Security
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronics Engineering, VIT-AP University, Amaravati 522237, AP, India
References (51)
Show source-provided references
1. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
2. Fighting against COVID-19: A novel deep learning model based on YOLO-v2 with ResNet-50 for medical face mask detection10.1016/j.scs.2020.102600 ↗
3. Mask wearing detection algorithm based on improved YOLOv3 in complex scenes
4. Improved algorithm of RetinaFace for natural scene mask wear detection
5. Mask detection algorithm based on improved YOLO lightweight network
6. Diwan, T., Anirudh, G., and Tembhurne, J.V. (2022). Object detection using YOLO: Challenges, architectural successors, datasets and applications. Multimed. Tools Appl., 1–33.10.1007/s11042-022-13644-y ↗
7. Jignesh Chowdary, G., Punn, N.S., Sonbhadra, S.K., and Agarwal, S. (2020, January 15–18). Face mask detection using transfer learning of inceptionv3. Proceedings of the International Conference on Big Data Analytics, Sonipat, India.10.1007/978-3-030-66665-1_6 ↗
8. A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic10.1016/j.measurement.2020.108288 ↗
9. Transfer learning for mobile real-time face mask detection and localization10.1093/jamia/ocab052 ↗
10. Bu, W., Xiao, J., Zhou, C., Yang, M., and Peng, C. (2017, January 19–21). A cascade framework for masked face detection. Proceedings of the 2017 IEEE International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM), Ningbo, China.10.1109/ICCIS.2017.8274819 ↗
11. Jiang, M., Fan, X., and Yan, H. (2020). Retinamask: A face mask detector. arXiv.
12. Face mask assistant: Detection of face mask service stage based on mobile phone10.1109/JSEN.2021.3061178 ↗
13. Eyiokur, F.I., Ekenel, H.K., and Waibel, A. (2021). A computer vision system to help prevent the transmission of COVID-19. arXiv.
14. Christa, G.H., Jesica, J., Anisha, K., and Sagayam, K.M. (2021, January 13–14). CNN-based mask detection system using openCV and MobileNetV2. Proceedings of the 2021 3rd International Conference on Signal Processing and Communication (ICSPC), Coimbatore, India.
15. Vinh, T.Q., and Anh, N.T.N. (2020, January 25–27). Real-time face mask detector using YOLOv3 algorithm and Haar cascade classifier. Proceedings of the 2020 International Conference on Advanced Computing and Applications (ACOMP), Quy Nhon, Vietnam.10.1109/ACOMP50827.2020.00029 ↗
16. A comparative analysis of face recognition models on masked faces
17. Zhang, E. (2021, January 20–22). A Real-Time Deep Transfer Learning Model for Facial Mask Detection. Proceedings of the 2021 Integrated Communications Navigation and Surveillance Conference (ICNS), Virtual.10.1109/ICNS52807.2021.9441582 ↗
18. Militante, S.V., and Dionisio, N.V. (2020, January 8). Real-time facemask recognition with alarm system using deep learning. Proceedings of the 2020 11th IEEE Control and System Graduate Research Colloquium (ICSGRC), Shah Alam, Malaysia.10.1109/ICSGRC49013.2020.9232610 ↗
19. Mandal, B., Okeukwu, A., and Theis, Y. (2021). Masked face recognition using resnet-50. arXiv.
20. Efficient masked face recognition method during the covid-19 pandemic10.1007/s11760-021-02050-w ↗
21. CNN-based bi-directional and directional long-short term memory network for determination of face mask10.1016/j.bspc.2021.103216 ↗
22. A novel GAN-based network for unmasking of masked face10.1109/ACCESS.2020.2977386 ↗
23. Maharani, D.A., Machbub, C., Rusmin, P.H., and Yulianti, L. (2020, January 14–15). Improving the capability of real-time face masked recognition using cosine distance. Proceedings of the 2020 6th International Conference on Interactive Digital Media (ICIDM), Virtual.10.1109/ICIDM51048.2020.9339677 ↗
24. Boutros, F., Damer, N., Kirchbuchner, F., and Kuijper, A. (2021). Unmasking face embeddings by self-restrained triplet loss for accurate masked face recognition. arXiv.10.1016/j.patcog.2021.108473 ↗
25. Deng, H., Feng, Z., Qian, G., Lv, X., Li, H., and Li, G. (2021). MFCosface: A masked-face recognition algorithm based on large margin cosine loss. Appl. Sci., 11.10.3390/app11167310 ↗
26. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
27. Towards NIR-VIS masked face recognition10.1109/LSP.2021.3071663 ↗
28. Geng, M., Peng, P., Huang, Y., and Tian, Y. (2020, January 12–16). Masked face recognition with generative data augmentation and domain constrained ranking. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.10.1145/3394171.3413723 ↗
29. Masked-face recognition using deep metric learning and FaceMaskNet-2110.1007/s10489-021-03150-3 ↗
30. Cropping and attention based approach for masked face recognition10.1007/s10489-020-02100-9 ↗
31. DSA-Face: Diverse and sparse attentions for face recognition robust to pose variation and occlusion10.1109/TIFS.2021.3109463 ↗
32. SSDMNV2: A real time DNN-based face mask detection system using single shot multibox detector and MobileNetV210.1016/j.scs.2020.102692 ↗
33. Li, C., Ge, S., Zhang, D., and Li, J. (2020, January 12–16). Look through masks: Towards masked face recognition with de-occlusion distillation. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.10.1145/3394171.3413960 ↗
34. Wang, Z., Wang, G., Huang, B., Xiong, Z., Hong, Q., Wu, H., Yi, P., Jiang, K., Wang, N., and Pei, Y. (2020). Masked face recognition dataset and application. arXiv.
35. Ding, F., Peng, P., Huang, Y., Geng, M., and Tian, Y. (2020, January 12–16). Masked face recognition with latent part detection. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.10.1145/3394171.3413731 ↗
36. A real time face mask detection system using convolutional neural network10.1007/s11042-022-12166-x ↗
37. Face mask detection using deep learning: An approach to reduce risk of Coronavirus spread10.1016/j.jbi.2021.103848 ↗
38. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
39. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.
40. Lightweight facenet based on mobilenet
41. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
42. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27–30). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.308 ↗
43. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017, January 4–9). Inception-v4, inception-resnet and the impact of residual connections on learning. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.11231 ↗
44. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
45. (2021, March 11). ImageNet. Available online: http://www.image-net.org.
46. Prajnasb (2020, April 14). Observations. Available online: https://github.com/prajnasb/observations.
47. Confusion Matrix
48. Hong, Q., Wang, Z., He, Z., Wang, N., Tian, X., and Lu, T. (2020, January 9–11). Masked face recognition with identification association. Proceedings of the 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI), Baltimore, MD, USA.10.1109/ICTAI50040.2020.00116 ↗
49. Anwar, A., and Raychowdhury, A. (2020). Masked face recognition for secure authentication. arXiv.
50. Farman, H., Khan, T., Khan, Z., Habib, S., Islam, M., and Ammar, A. (2022). Real-Time Face Mask Detection to Ensure COVID-19 Precautionary Measures in the Developing Countries. Appl. Sci., 12.10.3390/app12083879 ↗
51. Alzu’bi, A., Albalas, F., AL-Hadhrami, T., Younis, L.B., and Bashayreh, A. (2021). Masked Face Recognition Using Deep Learning: A Review. Electronics, 10.10.3390/electronics10212666 ↗