journal article·2025-11-23Open access

Document Image Verification Based on Paragraph Alignment and Subtle Change Detection

Daoquan Li, Weifei Jia, Quanlin Yu, Zhaoxu Hu

Applied Sciences· Volume 15· Issue 23

Abstract

The digitization of paper documents enables rapid sharing and long-term preservation of information, making it a widely adopted approach for efficient document storage and management across various domains. However, the recent advances in image editing software pose an increasing threat to the integrity of document images. Comparing the input with the corresponding reference document image is a direct and effective approach to verification. Nevertheless, this task is challenging due to two key factors, namely, the need for efficient retrieval of the reference document images and the difficulty of detecting subtle content changes under the print–scan (PS) distortions. To address these challenges, this work proposes a document image verification scheme based on paragraph alignment and subtle change detection. It first extracts paragraph structural features from both input and reference document images to achieve efficient image retrieval and accurate paragraph alignment. Based on the alignment results, the proposed scheme employs contrastive learning to reduce the effect of PS distortions in extracting features from the input and reference document images. Finally, an additional verification step is introduced that significantly reduces the false positive detection by addressing the feature misalignment within the extracted paragraphs. To evaluate the proposed scheme, extensive experiments were conducted on databases constructed from public datasets, and various benchmark methods were compared. Experimental results show that the proposed scheme outperforms benchmark methods, achieving an accuracy score of 0.963.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Handwritten Text Recognition Techniques · Image Retrieval and Classification Techniques · Advanced Image and Video Retrieval Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Information and Control Engineering, Qingdao University of Technology, Qingdao 266520, China

References (51)

Show source-provided references
1. Text-Line Examination for Document Forgery Detection10.1007/s10032-011-0181-5 ↗
2. Jain, R., and Doermann, D.S. (2015, January 23–26). Localized Document Image Change Detection. Proceedings of the International Conference on Document Analysis and Recognition (ICDAR’15), Tunis, Tunisia.
3. Park, D., Kim, S., Kim, M., Yarram, N.R., Joe, S., Gwon, Y., and Choi, J. (2023, January 8–11). Document Change Detection With Hierarchical Patch Comparison. Proceedings of the IEEE International Conference on Image Processing (ICIP’15), Kuala Lumpur, Malaysia.10.1109/ICIP49359.2023.10222407 ↗
4. Park, D., Yarram, N.R., Kim, S., Kim, M., Cho, S., and Lee, T. (March, January 26). Text Change Detection in Multilingual Documents Using Image Comparison. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV’24), Tucson, AZ, USA.
5. Document Recapture Detection Based on a Unified Distortion Model of Halftone Cells10.1109/TIFS.2022.3192999 ↗
6. Deep Learning Algorithms for Arabic Optical Character Recognition: A Survey10.61356/j.mawa.2024.26861 ↗
7. Jain, R., and Doermann, D.S. (2013, January 25–28). VisualDiff: Document Image Verification and Change Detection. Proceedings of the International Conference on Document Analysis and Recognition (ICDAR’13), Washington, DC, USA.
8. Survey of Post-OCR Processing Approaches10.1145/3453476 ↗
9. Neudecker, C., Baierer, K., Gerber, M., Clausner, C., Antonacopoulos, A., and Pletschacher, S. (2021, January 5–6). A Survey of OCR Evaluation Tools and Metrics. Proceedings of the International Workshop on Historical Document Imaging and Processing (HIP’21), Lausanne, Switzerland.10.1145/3476887.3476888 ↗
10. Segmentation of Text Lines Using Multi-Scale CNN from Warped Printed and Handwritten Document Images10.1007/s10032-021-00370-8 ↗
11. Gradient-Based Learning Applied to Document Recognition10.1109/5.726791 ↗
12. Beyond Document Object Detection: Instance-Level Segmentation of Complex Layouts10.1007/s10032-021-00380-6 ↗
13. A Keyword Retrieval System for Historical Mongolian Document Images10.1007/s10032-013-0203-6 ↗
14. Yan, H., and Watanabe, T. (2013, January 17–19). Document page retrieval based on geometric layout features. Proceedings of the International Conference on Ubiquitous Information Management and Communication (ICUIMC ’13), Kota Kinabalu, Malaysia.
15. Multi-Domain Intelligent System for Document Image Retrieval
16. Retrieval-Based Language Model Adaptation for Handwritten Chinese Text Recognition10.1007/s10032-022-00419-2 ↗
17. The Indexing and Retrieval of Document Images: A Survey10.1006/cviu.1998.0692 ↗
18. Information Retrieval from Documents: A Survey10.1023/A:1009950525500 ↗
19. CNN-Based Segmentation of Speech Balloons and Narrative Text Boxes from Comic Book Page Images10.1007/s10032-021-00366-4 ↗
20. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks10.1109/TPAMI.2016.2577031 ↗
21. Azmi, R., Akbari, M., Akbari, H., and Shirazi, H.R.G. (2010, January 22–24). LGL-DIR: Layout Graph for Layout Based Document Image Retrieval. Proceedings of the International Conference on Education Technology and Computer (ICETC’10), Shanghai, China.
22. Text Recognition for Vietnamese Identity Card Based on Deep Features Network10.1007/s10032-021-00363-7 ↗
23. Namboodiri, A.M., and Jain, A.K. (2007). Document Structure and Layout Analysis. Digital Document Processing: Major Directions and Recent Advances, Springer.10.1007/978-1-84628-726-8_2 ↗
24. Classification of document pages using structure-based features10.1007/PL00013566 ↗
25. Tang, Z., Yang, Z., Wang, G., Fang, Y., Liu, Y., Zhu, C., Zeng, M., Zhang, C.Y., and Bansal, M. (2023, January 18–22). Unifying Vision, Text, and Layout for Universal Document Processing. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.01845 ↗
26. Deep features based convolutional neural network model for text and non-text region segmentation from document images10.1016/j.asoc.2021.107917 ↗
27. An Improved Fingerprint-Based Document Image Retrieval Using Multi-Resolution Histogram of Oriented Gradient Features10.5829/IJE.2022.35.04A.15 ↗
28. Lombardi, F., and Marinai, S. (2020). Deep Learning for Historical Document Analysis and Recognition—A Survey. J. Imaging, 6.10.3390/jimaging6100110 ↗
29. van Strien, D.A., Beelen, K., Ardanuy, M.C., Hosseini, K., McGillivray, B., and Colavizza, G. (2020, January 22–24). Assessing the Impact of OCR Quality on Downstream NLP Tasks. Proceedings of the International Conference on Agents and Artificial Intelligence (ICAART’20), Valletta, Malta.10.5220/0009169004840496 ↗
30. Cheng, H., Zhang, P., Wu, S., Zhang, J., Zhu, Q., Xie, Z., Li, J., Ding, K., and Jin, L. (2023, January 17–24). M6Doc: A Large-Scale Multi-Format, Multi-Type, Multi-Layout, Multi-Language, Multi-Annotation Category Dataset for Modern Document Layout Analysis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’23), Vancouver, BC, Canada.10.1109/CVPR52729.2023.01453 ↗
31. Bay, H., Tuytelaars, T., and Gool, L.V. (2006, January 7–13). SURF: Speeded Up Robust Features. Proceedings of the European Conference on Computer Vision (ECCV’06), Graz, Austria.10.1007/11744023_32 ↗
32. Ke, Y., and Sukthankar, R. (2004, January 27). PCA-SIFT: A More Distinctive Representation for Local Image Descriptors. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’04), Washington, DC, USA.
33. Rublee, E., Rabaud, V., Konolige, K., and Bradski, G.R. (2011, January 6–13). ORB: An Efficient Alternative to SIFT or SURF. Proceedings of the International Conference on Computer Vision (ICCV’11), Barcelona, Spain.10.1109/ICCV.2011.6126544 ↗
34. Distinctive Image Features from Scale-Invariant Keypoints10.1023/B:VISI.0000029664.99615.94 ↗
35. Sequence-Aware Multimodal Page Classification of Brazilian Legal Documents
36. Shirdhonkar, M.S., and Kokare, M. (2011, January 15–17). Document Image Retrieval Using Signature as Query. Proceedings of the International Conference on Computer and Communication Technology (ICCCT’11), Allahabad, India.
37. Speeded-Up Robust Features (SURF)10.1016/j.cviu.2007.09.014 ↗
38. Da, C., Luo, C., Zheng, Q., and Yao, C. (2023, January 1–6). Vision Grid Transformer for Document Layout Analysis. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV’23), Paris, France.10.1109/ICCV51070.2023.01783 ↗
39. Segmentation for Document Layout Analysis: Not Dead Yet10.1007/s10032-021-00391-3 ↗
40. Extracting Text from Scanned Arabic Books: A Large-Scale Benchmark Dataset and A Fine-Tuned Faster-R-CNN Model10.1007/s10032-021-00382-4 ↗
41. Zhong, X., Tang, J., and Jimeno-Yepes, A. (2019, January 20–25). PubLayNet: Largest Dataset Ever for Document Layout Analysis. Proceedings of the International Conference on Document Analysis and Recognition (ICDAR’19), Sydney, NSW, Australia.10.1109/ICDAR.2019.00166 ↗
42. Biswas, S., Banerjee, A., Llad’os, J., and Pal, U. (2022). DocSegTr: An Instance-Level End-to-End Document Image Segmentation Transformer. arXiv.
43. Maity, S., Biswas, S., Manna, S., Banerjee, A., Llad’os, J., Bhattacharya, S., and Pal, U. (2023). SelfDocSeg: A Self-Supervised vision-based Approach towards Document Segmentation. arXiv.10.1007/978-3-031-41676-7_20 ↗
44. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
45. Liu, X., Chau, K.Y., Liu, X., and Wan, Y. (2023). The Progress of Smart Elderly Care Research: A Scientometric Analysis Based on CNKI and WOS. Int. J. Environ. Res. Public Health, 20.10.3390/ijerph20021086 ↗
46. Shekhar, R. (2013). Document Image Retrieval Using Bag of Visual Words Model. [Master’s Thesis, International Institute of Information Technology Hyderabad].10.1109/DAS.2012.96 ↗
47. Rahman, M.S., Rabbi, S.M.E., and Rashid, M.M. (2024). Optimizing Domain-Specific Image Retrieval: A Benchmark of FAISS and Annoy with Fine-Tuned Features. arXiv.
48. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the International Conference on Learning Representations (ICLR’15), San Diego, CA, USA.
49. Sandler, M., Howard, A.G., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18–22). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18), Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
50. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-Level Accuracy with 50× Fewer Parameters and <0.5 MB Model Size. arXiv.
51. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27–30). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16), Las Vegas, NV, USA.10.1109/CVPR.2016.308 ↗