An Ante Hoc Enhancement Method for Image-Based Complex Financial Table Extraction
Source abstract
Abstract
In the field of finance, the table is a common form of data organization. Extracting data from these tables in large quantities is a fundamental and important task for researchers. However, this can be a challenging task, as many tables exist in unstructured forms, such as scanned images in PDFs, rather than forms which can be easily processed, such as Excel spreadsheets. In recent years, a large number of table extraction methods utilizing heuristic algorithms or deep learning models have been proposed to free people from manual processing tasks, which are time-consuming and troublesome. Although existing methods achieve high levels of accuracy in processing some kinds of tables, they often fail to achieve optimal results when extracting complex financial tables with multi-line text and missing demarcation lines. In this article, we propose an enhancement method for image-based complex table extraction. This method consists of two modules: a split module and a filter module. The split module uses an OCR (optical character recognition) model to locate text regions, and a heuristic algorithm to obtain candidate demarcation lines. The filter module is based on a text semantic matching model and another heuristic algorithm. The experimental results show that the use of the proposed method can significantly improve the performance of different table extraction methods, with increases in F1 scores of between 5.10 and 14.36 points being recorded.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Currency Recognition and Detection · Handwritten Text Recognition Techniques · Image Retrieval and Classification Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Management, Wuhan University, Wuhan 430072, China
Intelligent Computing Laboratory for Cultural Heritage, Wuhan University, Wuhan 430072, China
Big Data Institute, Wuhan University, Wuhan 430072, China
Funding
Ministry of Science and Technology of the People’s Republic of China · National Demonstration Center for Experimental Library and Information Science Education, Wuhan University
References (41)
Show source-provided references
1. Itonori, K. (1993, January 20–22). Table structure recognition based on textblock arrangement and ruled line position. Proceedings of the 2nd International Conference on Document Analysis and Recognition (ICDAR’93), Tsukuba, Japan.
2. Chandran, S., and Kasturi, R. (1993, January 20–22). Structural recognition of tabulated data. Proceedings of the 2nd International Conference on Document Analysis and Recognition (ICDAR’93), Tsukuba, Japan.
3. Tabular document recognition10.1117/12.171096 ↗
4. Table recognition and understanding from pdf files10.1109/ICDAR.2007.4377094 ↗
5. Oro, E., and Ruffolo, M. (2009, January 26–29). Trex: An approach for recognizing and extracting tables from pdf documents. Proceedings of the 2009 10th International Conference on Document Analysis and Recognition, Barcelona, Spain.10.1109/ICDAR.2009.12 ↗
6. Kieninger, T., and Dengel, A. (1998, January 4–6). The t-recs table recognition and analysis system. Proceedings of the Document Analysis Systems: Theory and Practice: Third IAPR Workshop, DAS’98, Nagano, Japan. Selected Papers 3.
7. Kieninger, T., and Dengel, A. (2001, January 10–13). Applying the T-RECS table recognition system to the business letter domain. Proceedings of the Sixth International Conference on Document Analysis and Recognition, Seattle, WA, USA.
8. Deepdesrt: Deep learning for detection and structure recognition of tables in document images10.1109/ICDAR.2017.192 ↗
9. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
10. Siddiqui, S.A., Fateh, I.A., Rizvi, S.T.R., Dengel, A., and Ahmed, S. (2019, January 20–25). Deeptabstr: Deep learning based table structure recognition. Proceedings of the 2019 International Conference on Document Analysis and Recognition (ICDAR), Sydney, NSW, Australia.10.1109/ICDAR.2019.00226 ↗
11. Prasad, D., Gadpal, A., Kapadni, K., Visave, M., and Sultanpure, K. (2020, January 14–19). CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00294 ↗
12. Chi, Z., Huang, H., Xu, H.-D., Yu, H., Yin, W., and Mao, X.-L. (2019). Complicated table structure recognition. arXiv.
13. Li, C., Guo, R., Zhou, J., An, M., Du, Y., Zhu, L., Liu, Y., Hu, X., and Yu, D. (2022). Pp-structurev2: A stronger document analysis system. arXiv.
14. Ye, J., Qi, X., He, Y., Chen, Y., Gu, D., Gao, P., and Xiao, R. (2021). PingAn-VCGroup’s solution for ICDAR 2021 competition on scientific literature parsing task B: Table recognition to HTML. arXiv.
15. Wang, W., Xie, E., Li, X., Hou, W., Lu, T., Yu, G., and Shao, S. (2019, January 15–20). Shape robust text detection with progressive scale expansion network. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00956 ↗
16. Master: Multi-aspect non-local network for scene text recognition10.1016/j.patcog.2021.107980 ↗
17. Li, M., Zhang, L., Zhou, M., and Han, D. (2023). UTTSR: A novel non-structured text table recognition model powered by deep learning technology. Appl. Sci., 13.10.3390/app13137556 ↗
18. Liao, M., Wan, Z., Yao, C., Chen, K., and Bai, X. (2020, January 7–12). Real-time scene text detection with differentiable binarization. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i07.6812 ↗
19. Balsiger, D., Dimmler, H.-R., Egger-Horstmann, S., and Hanne, T. (2024). Assessing Large Language Models Used for Extracting Table Information from Annual Financial Reports. Computers, 13.10.3390/computers13100257 ↗
20. Li, Y., Luo, Y., Zheng, Y., Liu, G., and Gong, J. (2024). Research on Target Image Classification in Low-Light Night Vision. Entropy, 26.10.3390/e26100882 ↗
21. Split, embed and merge: An accurate table structure recognizer10.1016/j.patcog.2022.108565 ↗
22. Li, Y., Huang, Z., Yan, J., Zhou, Y., Ye, F., and Liu, X. (2021). GFTE: Graph-based financial table extraction. Pattern Recognition. ICPR International Workshops and Challenges: Virtual Event, 10–15 January 2021, Proceedings, Part II, Springer.10.1007/978-3-030-68790-8_50 ↗
23. The probabilistic relevance framework: BM25 and beyond10.1561/1500000019 ↗
24. Huang, P.-S., He, X., Gao, J., Deng, L., Acero, A., and Heck, L. (November, January 27). Learning deep structured semantic models for web search using clickthrough data. Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, San Francisco, CA, USA.
25. Shen, Y., He, X., Gao, J., Deng, L., and Mesnil, G. (2014, January 7–11). Learning semantic representations using convolutional neural networks for web search. Proceedings of the 23rd International Conference on World Wide Web, Seoul, Republic of Korea.10.1145/2567948.2577348 ↗
26. Wan, S., Lan, Y., Guo, J., Xu, J., Pang, L., and Cheng, X. (2016, January 12–17). A deep architecture for semantic matching with multiple positional sentence representations. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10342 ↗
27. Convolutional neural network architectures for matching natural language sentences
28. Deep sentence embedding using long short-term memory networks: Analysis and application to information retrieval10.1109/TASLP.2016.2520371 ↗
29. Reimers, N. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv.10.18653/v1/D19-1410 ↗
30. Pang, L., Lan, Y., Guo, J., Xu, J., Wan, S., and Cheng, X. (2016, January 12–17). Text matching as image recognition. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10341 ↗
31. Vaswani, A. (2017, January 4–9). Attention is all you need. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.
32. Devlin, J. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
33. Du, Y., Li, C., Guo, R., Yin, X., Liu, W., Zhou, J., Bai, Y., Yu, Z., Yang, Y., and Dang, Q. (2020). Pp-ocr: A practical ultra lightweight ocr system. arXiv.
34. Zheng, X., Burdick, D., Popa, L., Zhong, X., and Wang, N.X.R. (2021, January 5–9). Global table extractor (gte): A framework for joint table identification and cell structure recognition using visual context. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Virtual Conference.10.1109/WACV48630.2021.00074 ↗
35. Zhong, X., ShafieiBavani, E., and Jimeno Yepes, A. (2020). Image-based table recognition: Data, model, and evaluation. European Conference on Computer Vision, Springer.10.1007/978-3-030-58589-1_34 ↗
36. Sun, Y., Wang, S., Li, Y., Feng, S., Tian, H., Wu, H., and Wang, H. (2020, January 7–12). Ernie 2.0: A continual pre-training framework for language understanding. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i05.6428 ↗
37. Sun, Y., Wang, S., Feng, S., Ding, S., Pang, C., Shang, J., Liu, J., Chen, X., Zhao, Y., and Lu, Y. (2021). Ernie 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation. arXiv.
38. Pires, T. (2019). How multilingual is multilingual BERT. arXiv.10.18653/v1/P19-1493 ↗
39. Liu, Y. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv.
40. Liu, H., Li, X., Gong, M., Liu, B., Wu, Y., Jiang, D., Liu, Y., and Sun, X. (2024, January 20–27). Grab What You Need: Rethinking Complex Table Structure Recognition with Flexible Components Deliberation. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.10.1609/aaai.v38i4.28149 ↗
41. Smock, B., Pesala, R., and Abraham, R. (2022, January 18–24). PubTables-1M: Towards comprehensive table extraction from unstructured documents. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00459 ↗