TCSMAF: twin cascade spatial multi-scale attention filtering inpainting of traditional Chinese painting
Source abstract
Abstract
The preservation of cultural artifacts is vital for maintaining historical continuity, particularly for traditional Chinese paintings that often suffer from decay and damage over time. Existing inpainting methods struggle to simultaneously recover complex brushwork structures, maintain visual coherence, and preserve consistency across multiple resolutions. To address these challenges, we present the Twin Cascade Spatial Multi-scale Attention Filtering (TCSMAF) method, which adopts a symmetric multi-scale dual-branch architecture to capture complex structures and semantic details through parallel processing. A Spatial Kernel Module is proposed to enhance spatial perception by coordinating hierarchical features with spatial coordinate encoding. Moreover, a Multi-scale Spatial and Channel Attention module that adopts progressive convolution kernel sizes is introduced to improve texture reconstruction by leveraging features across different scales and channels. These technical innovations significantly advance digital inpainting methodologies, providing a robust framework specifically designed to handle the intricate textures and details of damaged paintings. The dataset and code are available at https://github.com/LPDLG/TCSMAF .
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Aesthetic Perception and Analysis · Computer Graphics and Visualization Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (53)
Show source-provided references
1. Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T. & Efros, A. A. Context encoders: Feature learning by inpainting. In Proc. IEEE conference on computer vision and pattern recognition, 2536–2544 (2016).10.1109/CVPR.2016.278 ↗
2. Li, W. et al. Mat: Mask-aware transformer for large hole image inpainting. In Proc. IEEE/CVF conference on computer vision and pattern recognition, 10758–10768 (2022).10.1109/CVPR52688.2022.01049 ↗
3. Dong, Q., Cao, C. & Fu, Y. Incremental transformer structure enhanced image inpainting with masking positional encoding. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11358–11368 (2022).10.1109/CVPR52688.2022.01107 ↗
4. Liu, H., Jiang, B., Xiao, Y. & Yang, C. Coherent semantic attention for image inpainting. In Proc. IEEE/CVF international conference on computer vision, 4170–4179 (2019).10.1109/ICCV.2019.00427 ↗
5. Zheng, H. et al. Image inpainting with cascaded modulation gan and object-aware training. In European Conference on Computer Vision, 277–296 (Springer, 2022).10.1007/978-3-031-19787-1_16 ↗
6. Iizuka, S., Simo-Serra, E. & Ishikawa, H. Globally and locally consistent image completion. ACM Trans. Graph. 36, 1–14 (2017).10.1145/3072959.3073659 ↗
7. Deng, X. & Yu, Y. Ancient mural inpainting via structure information guided two-branch model. Herit. Sci. 11, 131 (2023).10.1186/s40494-023-00972-x ↗
8. Chi, L., Jiang, B. & Mu, Y. Fast Fourier convolution. Adv. Neural Inf. Process. Syst. 33, 4479–4488 (2020).
9. Xu, Z. et al. Hierarchical painter: Chinese landscape painting restoration with fine-grained styles. Vis. Intell. 1, 19 (2023).10.1007/s44267-023-00021-y ↗
10. Lugmayr, A., Danelljan, M., Romero, A., Timofte, R. & Van Gool, L. Repaint: Inpainting using denoising diffusion probabilistic models. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11461–11471 (2022).10.1109/CVPR52688.2022.01117 ↗
11. Guo, Q. et al. Jpgnet: Joint predictive filtering and generative network for image inpainting. In Proc. 29th ACM International conference on multimedia, 386–394 (2021).10.1145/3474085.3475170 ↗
12. Sun, Z., Lei, Y. & Wu, X. Ancient paintings inpainting based on dual encoders and contextual information. Herit. Sci. 12, 266 (2024).10.1186/s40494-024-01391-2 ↗
13. Zaixuan, F., Wei, T., Hai, Q. & Jinjian, Y. Implementation of grouting and salts reduction treatments of cave 85 wall paintings. In Agnew, N. (ed.) Conservation of Ancient Sites on the Silk Road, 480–487 (Getty Conservation Institute, 2010).
14. Wang, X. & Li, X. Application of western modern conservation principles in chinese paper relics: a case study of the restoration of the Qing dynasty couplet. J. Cult. Relics Conserv. 42, 45–52 (2023).
15. Tan, J. & Liu, Y. A brief discussion on the restoration and conservation techniques of color-heavy silk paintings. J. Cult. Herit. 58, 123–135 (2024).
16. Zhang, W. & Li, H. Exploration and reflection in paper relic restoration: a case study of the Qing dynasty scroll “immortal beneath the pines". J. Conserv. Stud. 39, 88–95 (2023).
17. Chen, Y. & Wang, L. Discussion on the restoration methods of the painting Hongli Jian Gu Tu. Stud. Conserv. 69, 210–221 (2024).
18. He, Z., Zheng, D. & Chen, X. Conservation of the Cai Zhongqi Guan Cha Shuo Tie in the Nanjing Museum collection. In Proc. 7th Annual Academic Conference of the China Association for Preservation Technology of Cultural Relics, 150–154 (Science Press, 2012).
19. Bertalmio, M., Sapiro, G., Caselles, V. & Ballester, C. Image inpainting. In Proc. 27th Annual Conference on Computer Graphics and Interactive Techniques, 417–424 (2000).10.1145/344779.344972 ↗
20. Meng, X. The value of traditional and modern techniques in the conservation and restoration of paper-based cultural relics in museum collections. Cultural Relics Identification and Appreciation 42–45 (2022).
21. Giakoumis, I., Nikolaidis, N. & Pitas, I. Digital image processing techniques for the detection and removal of cracks in digitized paintings. IEEE Trans. Image Process. 15, 178–188 (2006).10.1109/TIP.2005.860311 ↗
22. Barnes, C., Shechtman, E., Finkelstein, A. & Goldman, D. B. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. (TOG) 28, 24 (2009).10.1145/1531326.1531330 ↗
23. Yu, J. et al. Generative image inpainting with contextual attention. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 5505–5514 (2018).10.1109/CVPR.2018.00577 ↗
24. Goodfellow, I. et al. Generative adversarial nets. Advances in Neural Information Processing Systems 27 (2014).
25. Ko, K. & Kim, C.-S. Continuously masked transformer for image inpainting. In Proc. IEEE/CVF International Conference on Computer Vision, 13169–13178 (2023).10.1109/ICCV51070.2023.01211 ↗
26. Ho, J., Jain, A. & Abbeel, P. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, vol. 33, 6840–6851 (2020).
27. Nagar, S., Bala, A. & Patnaik, S. A. Adaptation of the super resolution sota for art restoration in camera capture images. In 2023 International Conference on Emerging Trends in Computing and Information Technologies (ICETCI), 1–6 (2023).10.1109/ICETCI58599.2023.10331102 ↗
28. Lyu, Q., Zhao, N., Yang, Y., Gong, Y. & Gao, J. A diffusion probabilistic model for traditional Chinese landscape painting super-resolution. Nature (2023).10.1186/s40494-023-01123-y ↗
29. Zhu, L. et al. Leveraging diffusion knowledge for generative image compression with fractal frequency-aware band learning. In Proc. 33rd ACM International Conference on Multimedia (ACM MM), 4037–4046 (2025).
30. Lu, R. et al. Diffusion-based bit-depth expansion. In 2024 IEEE 26th International Workshop on Multimedia Signal Processing (MMSP), 1–6 (IEEE, 2024).10.1109/MMSP61759.2024.10743597 ↗
31. Mildenhall, B. et al. Burst denoising with kernel prediction networks. In Proc. IEEE conference on computer vision and pattern recognition, 2502–2510 (2018).10.1109/CVPR.2018.00265 ↗
32. Carbajal, G., Vitoria, P., Lezama, J. & Musé, P. Blind motion deblurring with pixel-wise kernel estimation via kernel prediction networks. IEEE Transactions on Computational Imaging (2023).10.1109/TCI.2023.3322012 ↗
33. Guo, Q. et al. Efficientderain: Learning pixel-wise dilation filtering for high-efficiency single-image deraining. In Proc. AAAI Conference on Artificial Intelligence, vol. 35, 1487–1495 (2021).10.1609/aaai.v35i2.16239 ↗
34. Li, X. et al. Misf: Multi-level interactive siamese filtering for high-fidelity image inpainting. In Proc. IEEE/CVF conference on computer vision and pattern recognition, 1869–1878 (2022).10.1109/CVPR52688.2022.00191 ↗
35. Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012).
36. Ronneberger, O., Fischer, P. & Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, proceedings, part III 18, 234–241 (Springer, 2015).10.1007/978-3-319-24574-4_28 ↗
37. Lempitsky, V., Vedaldi, A. & Ulyanov, D. Deep image prior. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9446–9454 (IEEE, 2018).10.1109/CVPR.2018.00984 ↗
38. Johnson, J., Alahi, A. & Fei-Fei, L. Perceptual losses for real-time style transfer and super-resolution 694–711 (2016).10.1007/978-3-319-46475-6_43 ↗
39. Suvorov, R. et al. Resolution-robust large mask inpainting with Fourier convolutions. In Proc. IEEE/CVF winter conference on applications of computer vision, 2149–2159 (2022).10.1109/WACV51458.2022.00323 ↗
40. Zhou, B., Lapedriza, A., Khosla, A., Oliva, A. & Torralba, A. Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40, 1452–1464 (2017).10.1109/TPAMI.2017.2723009 ↗
41. Kaggle. Painter by Numbers data set. (2016). https://kaggle.com/competitions/painter-by-numbers. Accessed: 2025-09-24.
42. Loshchilov, I. & Hutter, F. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR) (2019).
43. Sonka, M., Hlavac, V. & Boyle, R. Image Processing, Analysis, and Machine Vision (2014).
44. Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004).10.1109/TIP.2003.819861 ↗
45. Zhang, R., Isola, P., Efros, A. A., Shechtman, E. & Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 586–595 (IEEE, 2018).10.1109/CVPR.2018.00068 ↗
46. Yi, Z., Tang, Q., Azizi, S., Jang, D. & Xu, Z. Contextual residual aggregation for ultra high-resolution image inpainting. In Proc. IEEE/CVF conference on computer vision and pattern recognition, 7508–7517 (2020).10.1109/CVPR42600.2020.00753 ↗
47. Deng, Y., Hui, S., Meng, R., Zhou, S. & Wang, J. Hourglass attention network for image inpainting. In European conference on computer vision, 483–501 (Springer, 2022).10.1007/978-3-031-19797-0_28 ↗
48. Liu, W. et al. Coordfill: Efficient high-resolution image inpainting via parameterized coordinate querying. In Proc. AAAI Conference on Artificial Intelligence, vol. 37, 1746–1754 (2023).10.1609/aaai.v37i2.25263 ↗
49. Zeng, Y., Fu, J., Chao, H. & Guo, B. Aggregated contextual transformations for high-resolution image inpainting. IEEE Trans. Vis. Comput. Graph. 29, 3266–3280 (2022).10.1109/TVCG.2022.3156949 ↗
50. Song, Y. et al. Score-based generative modeling through stochastic differential equations. In Proc. International Conference on Learning Representations (ICLR) (2021).
51. Liu, H., Wang, Y., Qian, B., Wang, M. & Rui, Y. Structure matters: Tackling the semantic discrepancy in diffusion models for image inpainting. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024).10.1109/CVPR52733.2024.00768 ↗
52. Johnson, J., Alahi, A. & Fei-Fei, L. Perceptual losses for real-time style transfer and super-resolution. In Proc. Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, Part II 14, 694–711 (Springer, 2016).10.1007/978-3-319-46475-6_43 ↗
53. Sanakoyeu, A., Kotovenko, D., Lang, S. & Ommer, B. A style-aware content loss for real-time HD style transfer. In Proc. European conference on computer vision (ECCV), 698–714 (2018).10.1007/978-3-030-01237-3_43 ↗