A Joint Training Model for Face Sketch Synthesis
Source abstract
Abstract
The exemplar-based method is most frequently used in face sketch synthesis because of its efficiency in representing the nonlinear mapping between face photos and sketches. However, the sketches synthesized by existing exemplar-based methods suffer from block artifacts and blur effects. In addition, most exemplar-based methods ignore the training sketches in the weight representation process. To improve synthesis performance, a novel joint training model is proposed in this paper, taking sketches into consideration. First, we construct the joint training photo and sketch by concatenating the original photo and its sketch with a high-pass filtered image of their corresponding sketch. Then, an offline random sampling strategy is adopted for each test photo patch to select the joint training photo and sketch patches in the neighboring region. Finally, a novel locality constraint is designed to calculate the reconstruction weight, allowing the synthesized sketches to have more detailed information. Extensive experimental results on public datasets show the superiority of the proposed joint training model, both from subjective perceptual and the FaceNet-based face recognition objective evaluation, compared to existing state-of-the-art sketch synthesis methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Face recognition and analysis · Face and Expression Recognition · Generative Adversarial Networks and Image Synthesis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Division of Computer Science and Engineering, Chonbuk National University, Jeonju 54896, Korea
Center for Advanced Image and Information Technology, Chonbuk National University, Jeonju 54896, Korea
Funding
Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education
References (27)
Show source-provided references
1. Unified framework for face sketch synthesis10.1016/j.sigpro.2016.06.014 ↗
2. Adaptive representation-based face sketch-photo synthesis10.1016/j.neucom.2016.10.095 ↗
3. Wan, W., and Lee, H.J. (2017, January 14–16). FaceNet Based Face Sketch Recognition. Proceedings of the 2017 International Conference on Computational Science and Computational Intelligence, Las Vegas, NV, USA.10.1109/CSCI.2017.73 ↗
4. Zhang, Y., Wang, N., Zhang, S., Li, J., and Gao, X. (2016, January 8–16). Fast face sketch synthesis via KD-tree search. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherland.10.1007/978-3-319-46604-0_5 ↗
5. Data-driven vs. model-driven: Fast face sketch synthesis10.1016/j.neucom.2016.07.071 ↗
6. Face sketch recognition10.1109/TCSVT.2003.818389 ↗
7. Liu, Q., Tang, X., Jin, H., Lu, H., and Ma, S. (2005, January 20–25). A nonlinear approach for face sketch synthesis and recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA.
8. Song, Y., Bao, L., Yang, Q., and Yang, M. (2014, January 5–12). Real-time exemplar-based face sketch synthesis. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10599-4_51 ↗
9. Face photo-sketch synthesis and recognition10.1109/TPAMI.2008.222 ↗
10. Zhou, H., Kuang, Z., and Wong, K. (2012, January 18–20). Markov weight fields for face sketch synthesis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Rhode Island, USA.
11. Face sketch–photo synthesis and retrieval using sparse representation10.1109/TCSVT.2012.2198090 ↗
12. Multiple representations-based face sketch–photo synthesis10.1109/TNNLS.2015.2464681 ↗
13. Random sampling for fast face sketch synthesis10.1016/j.patcog.2017.11.008 ↗
14. Bayesian face sketch synthesis10.1109/TIP.2017.2651375 ↗
15. Zhang, L., Lin, L., Wu, X., Ding, S., and Zhang, L. (2015, January 23–26). End-to-end photo-sketch generation via fully convolutional representation learning. Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, Shanghai, China.10.1145/2671188.2749321 ↗
16. Content-adaptive sketch portrait generation by decompositional representation learning10.1109/TIP.2016.2623485 ↗
17. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8–13). Generative adversarial nets. Proceedings of the International Conference on Neural Information Processing Systems, Montreal, QC, Canada.
18. Graph-Regularized Locality-Constrained Joint Dictionary and Residual Learning for Face Sketch Synthesis10.1109/TIP.2018.2870936 ↗
19. Anchored neighborhood index for face sketch synthesis10.1109/TCSVT.2017.2709465 ↗
20. Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., and Gong, Y. (2010, January 13–18). Locality-constrained linear coding for image classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5540018 ↗
21. Zhang, W., Wang, X., and Tang, X. (2011, January 21–25). Coupled information-theoretic encoding for face photo-sketch recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA.10.1109/CVPR.2011.5995324 ↗
22. Tang, X., and Wang, X. (2002, January 22–25). Face photo recognition using sketch. Proceedings of the IEEE International Conference on Image Processing, New York, NY, USA.
23. Martinez, A., and Benavente, R. (1998). The AR Face Database, CVC. Technical Report.
24. Messer, K., Matas, J., Kittler, J., Luettin, J., and Maitre, G. (1999, January 22–24). XM2VTSDB: The extended M2VTS database. Proceedings of the International Conference on Audio- and Video-Based Biometric Person Authentication, Washington, DC, USA.
25. The FERET evaluation methodology for face recognition algorithms10.1109/34.879790 ↗
26. Chen, C., Tan, X., and Wong, K.K. (2018, January 12–15). Face sketch synthesis with style transfer using pyramid column feature. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, NV, USA.10.1109/WACV.2018.00059 ↗
27. Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 8–12). Facenet: A unified embedding for face recognition and clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298682 ↗