journal article·2023-07-04Open access

A Review of Document Image Enhancement Based on Document Degradation Problem

Yanxi Zhou, Shikai Zuo, Zhengxian Yang, Jinlong He, Jianwen Shi, Rui Zhang

Applied Sciences· Volume 13· Issue 13

Abstract

Document image enhancement methods are often used to improve the accuracy and efficiency of automated document analysis and recognition tasks such as character recognition. These document images could be degraded or damaged for various reasons including aging, fading handwriting, poor lighting conditions, watermarks, etc. In recent years, with the improvement of computer performance and the continuous development of deep learning, many methods have been proposed to enhance the quality of these document images. In this paper, we review six tasks of document degradation, namely, background texture, page smudging, fading, poor lighting conditions, watermarking, and blurring. We summarize the main models for each degradation problem as well as recent work, such as the binarization model that can be used to deal with the degradation of background textures, lettering smudges. When facing the problem of fading, a model for stroke connectivity can be used, while the other three degradation problems are mostly deep learning models. We discuss the current limitations and challenges of each degradation task and introduce the common public datasets and metrics. We identify several promising research directions and opportunities for future research.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Handwritten Text Recognition Techniques · Digital Media Forensic Detection · Vehicle License Plate Recognition

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Opto-Electronic and Communication Engineering, Xiamen University of Technology, Xiamen 361000, China

Funding

Natural Science Foundation of Fujian Province

References (85)

Show source-provided references
1. Robust document image binarization technique for degraded document images
2. Sulaiman, A., Omar, K., and Nasrudin, M.F. (2019). Degraded historical document binarization: A review on issues, challenges, techniques, and future directions. J. Imaging, 5.10.3390/jimaging5040048 ↗
3. Chen, X., He, X., Yang, J., and Wu, Q. (2011, January 20–25). An effective document image deblurring algorithm. Proceedings of the CVPR 2011, Washington, DC, USA.10.1109/CVPR.2011.5995568 ↗
4. Kligler, N., Katz, S., and Tal, A. (2018, January 19–23). Document enhancement using visibility detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00252 ↗
5. A new thresholding algorithm for document images based on the perception of objects by distance10.3233/ICA-130453 ↗
6. Historical document image restoration using multispectral imaging system10.1016/j.patcog.2012.12.015 ↗
7. Binarization of degraded document images based on contrast enhancement10.1007/s10032-018-0299-9 ↗
8. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
9. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
10. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
11. Anvari, Z., and Athitsos, V. (2019, January 5–7). A pipeline for automated face dataset creation from unlabeled images. Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments, New York, NY, USA.10.1145/3316782.3321522 ↗
12. Lin, W.-A., Chen, J.-C., Castillo, C.D., and Chellappa, R. (2018, January 19–23). Deep density clustering of unconstrained faces. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00848 ↗
13. Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 7–12). Facenet: A unified embedding for face recognition and clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298682 ↗
14. Gu, S., Zuo, W., Guo, S., Chen, Y., Chen, C., and Zhang, L. (2017, January 21–26). Learning dynamic guidance for depth image enhancement. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.83 ↗
15. Guo, C., Li, C., Guo, J., Loy, C.C., Hou, J., Kwong, S., and Cong, R. (2020, January 13–19). Zero-reference deep curve estimation for low-light image enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00185 ↗
16. Various image enhancement techniques—A critical review
17. Lin, Y.-H., Chen, W.-C., and Chuang, Y.-Y. (2020, January 13–19). Bedsr-net: A deep shadow removal network from a single document image. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01292 ↗
18. Hansen, P.C., Nagy, J.G., and O’leary, D.P. (2006). Deblurring Images: Matrices, Spectra, and Filtering, SIAM.10.1137/1.9780898718874 ↗
19. A threshold selection method from gray-level histograms10.1109/TSMC.1979.4310076 ↗
20. Adaptive document image binarization10.1016/S0031-3203(99)00055-2 ↗
21. Niblack, W. (1985). An Introduction to Digital Image Processing, Strandberg Publishing Company.
22. Efficient document image binarization using heterogeneous computing and parameter tuning10.1007/s10032-017-0293-7 ↗
23. Jana, P., Ghosh, S., Bera, S.K., and Sarkar, R. (2017, January 2–3). Handwritten document image binarization: An adaptive K-means based approach. Proceedings of the 2017 IEEE Calcutta Conference (CALCON), Kolkata, India.10.1109/CALCON.2017.8280729 ↗
24. Document binarization with automatic parameter tuning10.1007/s10032-012-0192-x ↗
25. Rani, U., Kaur, A., and Josan, G. (2020). Cognitive Computing in Human Cognition, Springer.
26. U-Net-bin: Hacking the document image binarization contest
27. Xiong, W., Yue, L., Zhou, L., Wei, L., and Li, M. (November, January 29). FD-Net: A Fully Dilated Convolutional Network for Historical Document Image Binarization. Proceedings of the Chinese Conference on Pattern Recognition and Computer Vision (PRCV), Beijing, China.
28. Binarization of degraded document images based on hierarchical deep supervised network10.1016/j.patcog.2017.08.025 ↗
29. GiB: A Game theory Inspired Binarization technique for degraded document images10.1109/TIP.2018.2878959 ↗
30. A selectional auto-encoder approach for document image binarization10.1016/j.patcog.2018.08.011 ↗
31. Document image binarization with cascaded generators of conditional generative adversarial networks10.1016/j.patcog.2019.106968 ↗
32. Peng, X., Wang, C., and Cao, H. (2019, January 20–25). Document binarization via multi-resolutional attention model with DRD loss. Proceedings of the 2019 International Conference on Document Analysis and Recognition (ICDAR), Sydney, Australia.10.1109/ICDAR.2019.00017 ↗
33. DE-GAN: A conditional generative adversarial network for document enhancement10.1109/TPAMI.2020.3022406 ↗
34. Degraded document image binarization using structural symmetry of strokes10.1016/j.patcog.2017.09.032 ↗
35. Bhunia, A.K., Bhunia, A.K., Sain, A., and Roy, P.P. (2019, January 22–25). Improving document binarization via adversarial noise-texture augmentation. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8803348 ↗
36. An enhanced binarization framework for degraded historical document images10.1186/s13640-021-00556-4 ↗
37. DP-LinkNet: A convolutional network for historical document image binarization
38. Chaurasia, A., and Culurciello, E. (2017, January 10–13). Linknet: Exploiting encoder representations for efficient semantic segmentation. Proceedings of the 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA.10.1109/VCIP.2017.8305148 ↗
39. Zhou, L., Zhang, C., and Wu, M. (2018, January 18–23). D-LinkNet: LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.10.1109/CVPRW.2018.00034 ↗
40. Two-stage generative adversarial networks for binarization of color document images10.1016/j.patcog.2022.108810 ↗
41. Souibgui, M.A., Biswas, S., Jemni, S.K., Kessentini, Y., Fornés, A., Lladós, J., and Pal, U. (2022). Docentr: An end-to-end document image enhancement transformer. arXiv.10.1109/ICPR56361.2022.9956101 ↗
42. Bako, S., Darabi, S., Shechtman, E., Wang, J., Sunkavalli, K., and Sen, P. (2016, January 20–24). Removing shadows from images of documents. Proceedings of the Asian Conference on Computer Vision, Taipei, Taiwan.10.1007/978-3-319-54187-7_12 ↗
43. Wang, J., Li, X., and Yang, J. (2018, January 19–23). Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00192 ↗
44. Jung, S., Hasan, M.A., and Kim, C. (2018, January 2–6). Water-filling: An efficient algorithm for digitized document shadow removal. Proceedings of the Asian Conference on Computer Vision, Perth, Australia.10.1007/978-3-030-20887-5_25 ↗
45. Chen, Z., Long, C., Zhang, L., and Xiao, C. (2021, January 11–17). Canet: A context-aware network for shadow removal. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00470 ↗
46. Shadow removal by a lightness-guided network with training on unpaired data10.1109/TIP.2020.3048677 ↗
47. Gangeh, M.J., Tiyyagura, S.R., Dasaratha, S.V., Motahari, H., and Duffy, N.P. (2019, January 14). Document enhancement system using auto-encoders. Proceedings of the Workshop on Document Intelligence at NeurIPS 2019, Vancouver, BC, Canada.
48. Enhance to read better: A multi-task adversarial network for handwritten document image enhancement10.1016/j.patcog.2021.108370 ↗
49. Liu, Y., Guo, M., Zhang, J., Zhu, Y., and Xie, X. (2019, January 21–25). A novel two-stage separable deep learning framework for practical blind watermarking. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.10.1145/3343031.3351025 ↗
50. Two-stage visible watermark removal architecture based on deep learning10.1049/iet-ipr.2020.0444 ↗
51. Mirza, M., and Osindero, S. (2014). Conditional generative adversarial nets. arXiv.
52. Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., and Paul Smolley, S. (2017, January 21–26). Least squares generative adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Honolulu, HI, USA.10.1109/ICCV.2017.304 ↗
53. Liu, Y., Zhu, Z., and Bai, X. (2021, January 3–8). Wdnet: Watermark-decomposition network for visible watermark removal. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV48630.2021.00373 ↗
54. Ge, S., Xia, Z., Fei, J., Sun, X., and Weng, J. (2022). A Robust Document Image Watermarking Scheme using Deep Neural Network. arXiv.10.1007/s11042-023-15048-y ↗
55. Hradiš, M., Kotera, J., Zemcık, P., and Šroubek, F. (2015, January 7–10). Convolutional neural networks for direct text deblurring. Proceedings of the BMVC, Swansea, UK.10.5244/C.29.6 ↗
56. Pan, J., Sun, D., Pfister, H., and Yang, M.-H. (2016, January 27–30). Blind image deblurring using dark channel prior. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.180 ↗
57. Zhu, J.-Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
58. Nah, S., Hyun Kim, T., and Mu Lee, K. (2017, January 21–26). Deep multi-scale convolutional neural network for dynamic scene deblurring. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.35 ↗
59. Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D., and Matas, J. (2018, January 19–23). Deblurgan: Blind motion deblurring using conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00854 ↗
60. Takano, N., and Alaghband, G. (2019). Srgan: Training dataset matters. arXiv.
61. Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6–11). Wasserstein generative adversarial networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia.
62. Blind deblurring of text images using a text-specific hybrid dictionary10.1109/TIP.2019.2933739 ↗
63. Lu, B., Chen, J.-C., and Chellappa, R. (2019, January 15–20). Unsupervised domain-specific deblurring via disentangled representations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.01047 ↗
64. Blind image deblurring via local maximum difference prior10.1109/ACCESS.2020.3039281 ↗
65. Blur2sharp: A gan-based model for document image deblurring10.2991/ijcis.d.210407.001 ↗
66. DeblurGAN-CNN: Effective Image Denoising and Recognition for Noisy Handwritten Characters10.1109/ACCESS.2022.3201560 ↗
67. Deng, F., Wu, Z., Lu, Z., and Brown, M.S. (2010, January 21–25). BinarizationShop: A user-assisted software suite for converting old documents to black-and-white. Proceedings of the 10th Annual Joint Conference on Digital Libraries, Gold Coast, Australia.10.1145/1816123.1816161 ↗
68. Hedjam, R., Nafchi, H.Z., Moghaddam, R.F., Kalacska, M., and Cheriet, M. (2015, January 23–26). Icdar 2015 contest on multispectral text extraction (ms-tex 2015). Proceedings of the 2015 13th International Conference on Document Analysis and Recognition (ICDAR), Tunis, Tunisia.10.1109/ICDAR.2015.7333947 ↗
69. Gatos, B., Ntirogiannis, K., and Pratikakis, I. (2009, January 26–29). ICDAR 2009 document image binarization contest (DIBCO 2009). Proceedings of the 2009 10th International Conference on Document Analysis and Recognition, Barcelona, Spain.10.1109/ICDAR.2009.246 ↗
70. Pratikakis, I., Gatos, B., and Ntirogiannis, K. (2011, January 18–21). ICDAR 2011 Document Image Binarization Contest (DIBCO 2011). Proceedings of the International Conference on Document Analysis & Recognition, Beijing, China.10.1109/ICDAR.2011.299 ↗
71. Pratikakis, I., Gatos, B., and Ntirogiannis, K. (2013, January 25–28). ICDAR 2013 document image binarization contest (DIBCO 2013). Proceedings of the 2013 12th International Conference on Document Analysis and Recognition, Washington, DC, USA.10.1109/ICDAR.2013.219 ↗
72. Pratikakis, I., Zagoris, K., Barlas, G., and Gatos, B. (2017, January 9–15). ICDAR2017 competition on document image binarization (DIBCO 2017). Proceedings of the 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, Japan.10.1109/ICDAR.2017.228 ↗
73. Pratikakis, I., Zagoris, K., Karagiannis, X., Tsochatzidis, L., Mondal, T., and Marthot-Santaniello, I. (2019, January 20–25). ICDAR 2019 Competition on Document Image Binarization (DIBCO 2019). Proceedings of the 2019 International Conference on Document Analysis and Recognition (ICDAR), Sydney, Australia.10.1109/ICDAR.2019.00249 ↗
74. Pratikakis, I., Gatos, B., and Ntirogiannis, K. (2010, January 16–18). H-DIBCO 2010-handwritten document image binarization competition. Proceedings of the 2010 12th International Conference on Frontiers in Handwriting Recognition, Washington, DC, USA.10.1109/ICFHR.2010.118 ↗
75. Pratikakis, I., Gatos, B., and Ntirogiannis, K. (2012, January 18–20). ICFHR 2012 competition on handwritten document image binarization (H-DIBCO 2012). Proceedings of the 2012 International Conference on Frontiers in Handwriting Recognition, Bari, Italy.10.1109/ICFHR.2012.216 ↗
76. Ntirogiannis, K., Gatos, B., and Pratikakis, I. (2014, January 1–4). ICFHR2014 competition on handwritten document image binarization (H-DIBCO 2014). Proceedings of the 2014 14th International Conference on Frontiers in Handwriting Recognition, Crete, Greece.10.1109/ICFHR.2014.141 ↗
77. Pratikakis, I., Zagoris, K., Barlas, G., and Gatos, B. (2016, January 23–26). ICFHR2016 handwritten document image binarization contest (H-DIBCO 2016). Proceedings of the 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), Shenzhen, China.10.1109/ICFHR.2016.0118 ↗
78. Pratikakis, I., Zagori, K., Kaddas, P., and Gatos, B. (2018, January 5–8). ICFHR 2018 Competition on Handwritten Document Image Binarization (H-DIBCO 2018). Proceedings of the 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), Niagara Falls, NY, USA.10.1109/ICFHR-2018.2018.00091 ↗
79. (2023, February 13). IEEE/Iet Electronic Library. Available online: https://ieeexplore.ieee.org/Xplore/home.jsp.
80. (2023, February 13). China National Knowledge Infrastructure. Available online: https://www.cnki.net/.
81. Performance evaluation methodology for historical document image binarization10.1109/TIP.2012.2219550 ↗
82. Distance-reciprocal distortion measure for binary document images10.1109/LSP.2003.821748 ↗
83. An omnifont open-vocabulary OCR system for English and Arabic10.1109/34.771314 ↗
84. Testing the correlation of word error rate and perplexity10.1016/S0167-6393(01)00041-3 ↗
85. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗