journal article·2023Open access

APST-Flow: A Reversible Network-Based Artistic Painting Style Transfer Method

Meng Wang, Yixuan Shao, Haipeng Liu

Computers, Materials & Continua· Volume 75· Issue 3

Abstract

In recent years, deep generative models have been successfully applied to perform artistic painting style transfer (APST). The difficulties might lie in the loss of reconstructing spatial details and the inefficiency of model convergence caused by the irreversible en-decoder methodology of the existing models. Aiming to this, this paper proposes a Flow-based architecture with both the en-decoder sharing a reversible network configuration. The proposed APST-Flow can efficiently reduce model uncertainty via a compact analysis-synthesis methodology, thereby the generalization performance and the convergence stability are improved. For the generator, a Flow-based network using Wavelet additive coupling (WAC) layers is implemented to extract multi-scale content features. Also, a style checker is used to enhance the global style consistency by minimizing the error between the reconstructed and the input images. To enhance the generated salient details, a loss of adaptive stroke edge is applied in both the global and local model training. The experimental results show that the proposed method improves PSNR by 5%, SSIM by 6.2%, and decreases Style Error by 29.4% over the existing models on the ChipPhi set. The competitive results verify that APST-Flow achieves high-quality generation with less content deviation and enhanced generalization, thereby can be further applied to more APST scenes.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Computer Graphics and Visualization Techniques · 3D Shape Modeling and Analysis

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (54)

Show source-provided references
1. Imagenet large scale visual recognition challenge10.1007/s11263-015-0816-y ↗
2. The evolution of computing: AlphaGo10.1109/MCSE.2016.74 ↗
3. Local similarity-based spatial-spectral fusion hyperspectral image classification with deep CNN and Gabor filtering10.1109/TGRS.2021.3090410 ↗
4. A. Bochkovskiy, C. -Y. Wang and H. -Y. M. Liao, “Yolov4: Optimal speed and accuracy of object detection,” arXiv preprint arXiv:2004.10934, 2020.
5. H. Cate, F. Dalvi and Z. Hussain, “Deepface: Face generation using deep learning,” arXiv preprint arXiv:1701.01876, 2017.
6. Recommendation system using feature extraction and pattern recognition in clinical care systems10.1080/17517575.2018.1557256 ↗
7. End-to-end Chinese landscape painting creation using generative adversarial networks
8. ChipGAn: A generative adversarial network for Chinese ink wash painting style transfer
9. Detail-preserving CycleGAN-AdaIN framework for image-to-ink painting translation10.1109/ACCESS.2020.3009470 ↗
10. Multi-watermarking algorithm for medical image based on KAZE-DCT10.1007/s12652-021-03539-5 ↗
11. Towards photo watercolorization with artistic verisimilitude10.1109/TVCG.2014.2303984 ↗
12. Perceptual losses for real-time style transfer and super-resolution
13. Evolvement constrained adversarial learning for video style transfer
14. D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
15. Transform a simple sketch to a chinese painting by a multiscale deep neural network10.3390/a11010004 ↗
16. Image quilting for texture synthesis and transfer
17. Image-based color ink diffusion rendering10.1109/TVCG.2007.41 ↗
18. Texture synthesis using convolutional neural networks
19. L. A. Gatys, A. S. Ecker and M. Bethge, “A neural algorithm of artistic style,” arXiv preprint arXiv:1508.06576, 2015.10.1167/16.12.326 ↗
20. SegNet: A deep convolutional encoder-decoder architecture for image segmentation10.1109/TPAMI.2016.2644615 ↗
21. Fully convolutional networks for semantic segmentation
22. Arbitrary style transfer in real-time with adaptive instance normalization
23. Arbitrary style transfer with style-attentional networks
24. AdaAttN: Revisit attention mechanism in arbitrary neural style transfer
25. Universal style transfer via feature transforms
26. Neural abstract style transfer for chinese traditional painting
27. Fast style transfer for chinese tranditional ink painting
28. J. Cao, Y. Hong and L. Niu, “Painterly image harmonization in dual domains,” arXiv preprint arXiv:2212.08846, 2022.
29. L. Dinh, D. Krueger and Y. Bengio, “Nice: Non-linear independent components estimation,” arXiv preprint arXiv:1410.8516, 2014.
30. L. Dinh, J. Sohl-Dickstein and S. Bengio, “Density estimation using real NVP,” arXiv preprint arXiv:1605.08803, 2016.
31. Glow: Generative flow with invertible 1 × 1 convolutions
32. Flow++: Improving flow-based generative models with variational dequantization and architecture design
33. BDCN: Bi-directional cascade network for perceptual edge detection10.1109/TPAMI.2020.3007074 ↗
34. IsGAN: Identity-sensitive generative adversarial network for face photo-sketch synthesis10.1016/j.patcog.2021.108077 ↗
35. Generative adversarial learning for detail-preserving face sketch synthesis10.1016/j.neucom.2021.01.050 ↗
36. Content-adaptive sketch portrait generation by decompositional representation learning10.1109/TIP.2016.2623485 ↗
37. Toward realistic face photo-sketch synthesis via composition-aided GANs10.1109/TCYB.2020.2972944 ↗
38. Back projection: An effective postprocessing method for GAN-based face sketch synthesis10.1016/j.patrec.2017.06.012 ↗
39. Style transfer for anime sketches with enhanced residual U-net and auxiliary classifier GAN
40. Dualast: Dual style-learning networks for artistic style transfer
41. Drafting and revision: Laplacian pyramid network for fast high-quality artistic style transfer
42. Structured output learning with conditional generative flows
43. Srflow: Learning the super-resolution space with normalizing flow
44. Wavelet flow: Fast training of high resolution normalizing flows
45. Beautyglow: On-demand makeup transfer framework with reversible generative network
46. Unbiased image style transfer via reversible neural flows
47. Image-to-image translation with conditional adversarial networks
48. Holistically-nested edge detection
49. Unpaired image-to-image translation using cycle-consistent adversarial networks
50. One-sided unsupervised domain mapping
51. Training generative adversarial networks with limited data
52. Face photo-sketch synthesis and recognition10.1109/TPAMI.2008.222 ↗
53. AesUST: Towards aesthetic-enhanced universal style transfer
54. Z. Wang, L. Zhao, Z. Zuo, A. Li, H. Chen et al., “MicroAST: Towards super-fast ultra-resolution arbitrary style transfer,” arXiv preprint arXiv:2211.15313, 2022.