Restoration of ancient Japanese manuscripts via the diffusion denoising restoration model and color space-based masking
Source abstract
Abstract
Ancient Japanese manuscripts are invaluable cultural assets but are often degraded by stains, fading, and bleed-through. We propose a restoration framework integrating the Diffusion Denoising Restoration Model (DDRM) with a two-stage mask generation process combining Automatic Color Equalization (ACE) and Gaussian Mixture Model clustering across multiple color spaces. A key novelty is the use of binarized text masks as guidance signals for DDRM through noise masking, which constrains denoising to character shapes and suppresses background noise. A high-resolution patch-based strategy with feather blending further enables seamless reconstruction of both text and background. Experiments on synthetic degradations of The Pillow Book and real manuscripts such as Tsurezuregusa demonstrate significant improvements over binarization-based DDRM, with an average PSNR gain of 10 dB, SSIM increase of 0.17, and LPIPS reduction from 0.49 to 0.28. The framework enhances textual clarity, preserves red annotations, and offers a scalable solution for AI-driven cultural heritage preservation.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Cultural Heritage Materials Analysis · Image Processing and 3D Reconstruction
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (42)
Show source-provided references
1. UNESCO. Memory of the World Programme. https://www.unesco.org/en/memory-world (2025).
2. Europeana Foundation. Europeana Collections. https://www.europeana.eu/en (2025).
3. Archives Portal Europe. Archives Portal Europe. https://www.archivesportaleurope.net (2025).
4. Akama, R. Digitization of classical Japanese books by Ritsumeikan University Art Research Center: On the ARC international model. J. Inf. Sci. Technol. 65, 181–186 (2015).
5. Clanuwat, T., Lamb, A. & Kitamoto, A. End-to-end pre-modern Japanese character (Kuzushiji) spotting with deep learning. In Proc. IPSJ SIG Computers and the Humanities Symposium, 15–20 (Information Processing Society of Japan, 2018).
6. Clanuwat, T., Lamb, A. & Kitamoto, A. KuroNet: Pre-modern Japanese Kuzushiji Character Recognition with Deep Learning. In Proc.15th International Conference on Document Analysis and Recognition (ICDAR,19) 607–614, https://doi.org/10.1109/ICDAR.2019.00104 (IEEE, 2019).10.1109/ICDAR.2019.00104 ↗
7. Lamb, A., Clanuwat, T. & Kitamoto, A. KuroNet: regularized residual U-Nets for end-to-end Kuzushiji character recognition. SN Comput. Sci. 1, 177 (2020).10.1007/s42979-020-00186-z ↗
8. Fiorucci, M. et al. Machine learning for cultural heritage: a survey. Pattern Recognit. Lett. 133, 102–108 (2020).10.1016/j.patrec.2020.02.017 ↗
9. Karadag, I. Machine learning for conservation of architectural heritage. Open House Int. 48, 23–37 (2023).10.1108/OHI-05-2022-0124 ↗
10. Stoean, R., Bacanin, N., Stoean, C. & Ionescu, L. Bridging the past and present: AI-driven 3D restoration of degraded artefacts for museum digital display. J. Cult. Herit. 69, 18–26 (2024).10.1016/j.culher.2024.07.008 ↗
11. Janković, R. Machine learning models for cultural heritage image classification: comparison based on attribute selection. Information 11, 12 (2019).10.3390/info11010012 ↗
12. Cosovic, M., Jankovic, R. & Ramic-Brkic, B. Cultural heritage image classification. In Proc. Data Analytics for Cultural Heritage: Current Trends and Concepts 25–45 (Springer, 2020).10.1007/978-3-030-66777-1_2 ↗
13. Grilli, E. & Remondino, F. Classification of 3D digital heritage. Remote Sens. 11, 847 (2019).10.3390/rs11070847 ↗
14. Belhi, A., Foufou, S., Bouras, A. & Sadka, A. H. Digitization and preservation of cultural heritage products. In Proc. IFIP Int. Conf. Product Lifecycle Management 241–253 (Springer, 2017).10.1007/978-3-319-72905-3_22 ↗
15. Buragohain, D., Meng, Y., Deng, C., Li, Q. & Chaudhary, S. Digitalizing cultural heritage through metaverse applications: Challenges, opportunities, and strategies. Herit. Sci. 12, 295 (2024).10.1186/s40494-024-01403-1 ↗
16. Tai, N. Digital archiving of the spatial experience of cultural heritage sites with ancient lighting. J. Comput. Cult. Herit. 16, 45 (2023).10.1145/3589229 ↗
17. Otsu, N. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9, 62–66 (1979).10.1109/TSMC.1979.4310076 ↗
18. Sauvola, J. & Pietikäinen, M. Adaptive document image binarization. Pattern Recognit. 33, 225–236 (2000).10.1016/S0031-3203(99)00055-2 ↗
19. Jain, A. K. Fundamentals of Digital Image Processing (Prentice-Hall, 1989).
20. Yang, Z. et al. Summary of document image binarization Preprints 1, https://doi.org/10.20944/preprints202401.1534.v1 (2024).10.20944/preprints202401.1534.v1 ↗
21. Hanif, M., Tonazzini, A., Hussain, S. F., Khalil, A. & Habib, U. Restoration and content analysis of ancient manuscripts via color space based segmentation. PLoS ONE 18, 1–14 (2023).10.1371/journal.pone.0282142 ↗
22. Bianco, G. et al. A framework for virtual restoration of ancient documents by combination of multispectral and 3D imaging. In Proc. Eurographics Italian Chapter Conference 1–7 (DBLP, 2010).
23. Munoz-Pandiella, I., Andujar, C., Cayuela, B., Pueyo, X. & Bosch, C. Automated digital color restitution of mural paintings using minimal art historian input. In Proc. Eurographics Workshop on Graphics and Cultural Heritage 316–325 (Computers & Graphics, 2023).10.1016/j.cag.2023.06.021 ↗
24. Kaneko, H., Ishibashi, R. & Meng, L. Deteriorated characters restoration for early Japanese books using enhanced CycleGAN. Heritage 6, 4345–4361 (2023).10.3390/heritage6050230 ↗
25. Wenjun, Z., Benpeng, S., Ruiqi, F., Xihua, P. & Shanxiong, C. EA-GAN: restoration of text in ancient Chinese books based on an example attention generative adversarial network. Herit. Sci. 11, 882 (2023).10.1186/s40494-023-00882-y ↗
26. Chen, S., Yang, Y., Liu, X. & Zhu, S. Dual discriminator GAN: restoring ancient Yi characters. ACM Trans. Asian Low-Resour. Lang. Inf. Process. 21, 66 (2022).10.1145/3490031 ↗
27. Yalin, M., Li, L., Yichun, J. & Li, G. Research on denoising method of Chinese ancient character image based on Chinese character writing standard model. Sci. Rep. 12, 19795 (2022).10.1038/s41598-022-24388-y ↗
28. Sun, G., Zheng, Z. & Zhang, M. End-to-end rubbing restoration using generative adversarial networks. Preprint at arXiv https://doi.org/10.48550/arXiv.2205.03743 (2022).10.48550/arXiv.2205.03743 ↗
29. Ho, J., Jain, A. & Abbeel, P. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, vol. 33, 6840–6851 (NeurIPS, 2020).
30. Song, J., Meng, C. & Ermon, S. Denoising diffusion implicit models. In Proc. 9th International Conference on Learning Representations (ICLR 2021).
31. Li, H. et al. Towards automated Chinese ancient character restoration: a diffusion-based method with a new dataset. In Proc. AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence, 342–351 (AAAI Press, 2024).10.1609/aaai.v38i4.28090 ↗
32. Chen, Y., Zhang, A., Gao, F. & Guo, J. Ancient mural super-resolution reconstruction based on conditional diffusion model for enhanced visual information. npj Herit. Sci. 13, 205 (2025).10.1038/s40494-025-01718-7 ↗
33. Wang, Y., Xiao, M., Hu, Y., Yan, J. & Zhu, Z. Research on restoration of murals based on diffusion model and transformer. Comput. Mater. Continua 80, 4433–4449 (2024).10.32604/cmc.2024.053232 ↗
34. Zhao, F., Ren, H., Su, Z., Zhu, X. & Zhang, C. Diffusion-based heterogeneous network for ancient mural restoration. npj Herit. Sci. 13, 206 (2025).10.1038/s40494-025-01719-6 ↗
35. Kaneko, H., Yoshizu, Y., Ishibashi, R., Meng, L. & Deng, M. An attempt at zero-shot ancient documents restoration based on diffusion models. In Proc. 2023 International Conference on Advanced Mechatronic Systems (ICAMechS) 1–6 (IEEE, 2023).10.1109/ICAMechS59878.2023.10272811 ↗
36. Yoshizu, Y., Kaneko, H., Ishibashi, R., Meng, L. & Deng, M. Two-stage stains reduction with diffusion models for ancient Japanese manuscripts. In Proc. 2024 International Conference on Cyber-Physical Social Intelligence (ICCSI) 1–6 (IEEE, 2024).10.1109/ICCSI62669.2024.10799366 ↗
37. Rizzi, A., Gatta, C. & Marini, D. A new algorithm for unsupervised global and local color correction. Pattern Recognit. Lett. 24, 1663–1677 (2003).10.1016/S0167-8655(02)00323-9 ↗
38. Huynh-Thu, Q. & Ghanbari, M. Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 44, 800–801 (2008).10.1049/el:20080522 ↗
39. Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004).10.1109/TIP.2003.819861 ↗
40. Zhang, R., Isola, P., Efros, A. A., Shechtman, E. & Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 586–595 (IEEE, 2018).10.1109/CVPR.2018.00068 ↗
41. Güzelci, O. Z., Alaçam, S., Bekiroğlu, B. & Karadag, I. A machine learning-based prediction model for architectural heritage: The case of domed Sinan mosques. Digit. Appl. Archaeol. Cult. Herit. 35, e00370 (2024).
42. Güzelci, O. Z. A machine learning-based model to predict the cap geometry of Anatolian Seljuk Kümbets. Period. Polytech. Archit. 53, 207–219 (2022).10.3311/PPar.20112 ↗