journal article·2019-12-19Open access

Progressive Full Data Convolutional Neural Networks for Line Extraction from Anime-Style Illustrations

Yuchen Xin, Hon-Cheng Wong, Sio-Long Lo, Junliang Li

Applied Sciences· Volume 10· Issue 1

Abstract

Anime-style comics are popular world-wide and an important industry in Asia. However, the output quantity and quality control of art workers have become the biggest obstacle to industrialization, and it is time consuming to produce new manga without the help of an intelligence assisted tool. As deep learning techniques have achieved great successes in different areas, it is worth exploring them to develop algorithms and systems for computational manga. Extracting line drawings from finished illustrations is one of the main tasks in drawing a manuscript and also a crucial task in the common painting process. However, traditional filters such as Sobel, Laplace, and Canny cannot output good results and require manual adjustments of the parameters. In order to address these problems, in this paper, we propose progressive full data convolutional neural networks for extracting lines from anime-style illustrations. Experimental results show that our progressive full data convolutional neural networks not only can learn as much as processing methods for the detailed regions, but also can accomplish the target extraction work with only a small training dataset.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Handwritten Text Recognition Techniques · Computer Graphics and Visualization Techniques · Image Processing and 3D Reconstruction

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Faculty of Information Technology, Macau University of Science and Technology, Macao, China

Funding

The Science and Technology Development Fund of Macao SAR

References (33)

Show source-provided references
1. A Computational Approach to Edge Detection10.1109/TPAMI.1986.4767851 ↗
2. Kang, H., Lee, S., and Chui, C.K. (2007, January 4–5). Coherent Line Drawing. Proceedings of the 5th International Symposium on Non-photorealistic Animation and Rendering, NPAR ’07, San Diego, CA, USA.10.1145/1274871.1274878 ↗
3. Tomasi, C., and Manduchi, R. (1998, January 4–7). Bilateral Filtering for Gray and Color Images. Proceedings of the Sixth International Conference on Computer Vision, ICCV ’98, Bombay, India.
4. Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing10.1023/A:1025384832106 ↗
5. Structure Extraction from Texture via Relative Total Variation10.1145/2366145.2366158 ↗
6. Ito, K., Matsui, Y., Yamasaki, T., and Aizawa, K. (2015). Separation of Manga Line Drawings and Screentones. Eurographics 2015 Short Paper.
7. Digital Reconstruction of Halftoned Color Comics10.1145/2366145.2366159 ↗
8. Manga Vectorization and Manipulation with Procedural Simple Screentone10.1109/TVCG.2016.2525774 ↗
9. Mastering Sketching: Adversarial Augmentation for Structured Prediction
10. Region-based structure line detection for cartoons10.1007/s41095-015-0007-3 ↗
11. Deep Extraction of Manga Structural Lines10.1145/3072959.3073675 ↗
12. Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Springer International Publishing.10.1007/978-3-319-24571-3 ↗
13. Closure-aware Sketch Simplification10.1145/2816795.2818067 ↗
14. Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup
15. Fully Convolutional Networks for Semantic Segmentation10.1109/TPAMI.2016.2572683 ↗
16. Let There Be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification10.1145/2897824.2925974 ↗
17. Real-time User-guided Image Colorization with Learned Deep Priors10.1145/3072959.3073703 ↗
18. Deep Unsupervised Pixelization10.1145/3272127.3275082 ↗
19. Dynamic Manga: Animating Still Manga via Camera Movement10.1109/TMM.2016.2609415 ↗
20. Automatic Stylistic Manga Layout10.1145/2366145.2366160 ↗
21. Look over Here: Attention-directing Composition of Manga Elements10.1145/2601097.2601183 ↗
22. Pang, X., Cao, Y., Lau, R.W., and Chan, A.B. (2014, January 3–7). A Robust Panel Extraction Method for Manga. Proceedings of the 22Nd ACM International Conference on Multimedia, MM ’14, Orlando, FL, USA.10.1145/2647868.2654990 ↗
23. Text-aware balloon extraction from manga10.1007/s00371-015-1084-0 ↗
24. Manga Colorization10.1145/1141911.1142017 ↗
25. Richness-preserving Manga Screening10.1145/1409060.1409108 ↗
26. Ci, Y., Ma, X., Wang, Z., Li, H., and Luo, Z. (2018, January 22–26). User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks. Proceedings of the 2018 ACM Multimedia Conference, Seoul, Korea.10.1145/3240508.3240661 ↗
27. Hensman, P., and Aizawa, K. (2017). cGAN-based Manga Colorization Using a Single Training Image. arXiv.10.1109/ICDAR.2017.295 ↗
28. Globally Optimal Toon Tracking10.1145/2897824.2925872 ↗
29. Stereoscopizing Cel Animations10.1145/2508363.2508396 ↗
30. ImageNet Classification with Deep Convolutional Neural Networks10.1145/3065386 ↗
31. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
32. Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., and Weinberger, K.Q. (2014). Generative Adversarial Nets. Advances in Neural Information Processing Systems 27, Curran Associates, Inc.
33. Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014, January 6–12). Visualizing and Understanding Convolutional Networks. Proceedings of the Computer Vision—ECCV 2014, Zurich, Switzerland.10.1007/978-3-319-10602-1 ↗