Web Page Content Block Identification with Extended Block Properties
Source abstract
Abstract
Web page segmentation is one of the most influential factors for the automated integration of web page content with other systems. Existing solutions are focused on segmentation but do not provide a more detailed description of the segment including its range (minimum and maximum HTML code bounds, covering the segment content) and variants (the same segments with different content). Therefore the paper proposes a novel solution designed to find all web page content blocks and detail them for further usage. It applies text similarity and document object model (DOM) tree analysis methods to indicate the maximum and minimum ranges of each identified HTML block. In addition, it indicates its relation to other blocks, including hierarchical as well as sibling blocks. The evaluation of the method reveals its ability to identify more content blocks in comparison to human labeling (in manual labeling only 24% of blocks were labeled). By using the proposed method, manual labeling effort could be reduced by at least 70%. Better performance was observed in comparison to other analyzed web page segmentation methods, and better recall was achieved due to focus on processing every block present on a page, and providing a more detailed web page division into content block data by presenting block boundary range and block variation data.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Web Data Mining and Analysis · Algorithms and Data Compression · Text and Document Classification Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Information Technologies, Vilnius Gediminas Technical University, Sauletekio al. 11, LT-10223 Vilnius, Lithuania
References (41)
Show source-provided references
1. Xie, W., Zheng, W., Tang, P., and Ting, Y. (2022, January 15–17). Design and Implementation of Web Information Extraction System Based on Crawler. Proceedings of the 2nd International Conference on Electronic Materials and Information Engineering (EMIE 2022), Hangzhou, China.
2. Machine learning techniques in Web content mining: A comparative analysis
3. Cheng, S.C., and Lu, C.A. (2019, January 7–10). Retrieving Articles and Image Labeling Based on Relevance of Keywords. Proceedings of the 2019 International Conference on Machine Learning and Cybernetics (ICMLC), Kobe, Japan.10.1109/ICMLC48188.2019.8949205 ↗
4. Web page classification: A survey of perspectives, gaps, and future directions10.1007/s11042-019-08373-8 ↗
5. Autonomous schema markups based on intelligent computing for search engine optimization10.7717/peerj-cs.1163 ↗
6. Algorithms for Table Structure Recognition
7. Identifying informative web content blocks using web page segmentation
8. Web page segmentation and informative content extraction for effective information retrieval
9. Purely vision-based segmentation of web pages for assistive technology10.1016/j.cviu.2016.02.007 ↗
10. Box clustering segmentation: A new method for vision-based web page preprocessing10.1016/j.ipm.2017.02.002 ↗
11. Sanoja, A., and Gançarski, S. (2015, January 13–17). Web page segmentation evaluation. Proceedings of the 30th Annual ACM Symposium on Applied Computing, Salamanca, Spain.10.1145/2695664.2695786 ↗
12. Kiesel, J., Meyer, L., Kneist, F., Stein, B., and Potthast, M. (April, January 28). An empirical comparison of web page segmentation algorithms. Proceedings of the European Conference on Information Retrieval, Lucca, Italy.
13. Automatic Extraction of Web Page Text Information Based on Network Topology Coincidence Degree10.1155/2022/9220661 ↗
14. Adaptive segmented webpage text based malicious website detection10.1016/j.comnet.2022.109236 ↗
15. Integrated-Block: A New Combination Model to Improve Web Page Segmentation
16. Kohlschütter, C., and Nejdl, W. (2008, January 26). A densitometric approach to web page segmentation. Proceedings of the 17th ACM Conference on Information and Knowledge Management, Napa Valley, CA, USA.10.1145/1458082.1458237 ↗
17. Xiang, P., Yang, X., and Shi, Y. (2007, January 2–5). Web page segmentation based on gestalt theory. Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, Beijin, China.10.1109/ICME.2007.4285135 ↗
18. Alcic, S., and Conrad, S. (2011, January 25–27). Page segmentation by web content clustering. Proceedings of the International Conference on Web Intelligence, Mining and Semantics, Sogndal, Norway.10.1145/1988688.1988717 ↗
19. Language independent web news extraction system based on text detection framework10.1016/j.ins.2015.12.025 ↗
20. Sanoja, A., and Gançarski, S. (2017, January 24–27). Migrating web archives from html4 to html5: A block-based approach and its evaluation. Proceedings of the European Conference on Advances in Databases and Information Systems, Nicosia, Cyprus.10.1007/978-3-319-66917-5_25 ↗
21. Manugunta, R.K., Maskeliūnas, R., and Damaševičius, R. (2022). Deep Learning Based Semantic Image Segmentation Methods for Classification of Web Page Imagery. Future Internet, 14.10.3390/fi14100277 ↗
22. Griazev, K., and Ramanauskaitė, S. (2021). Multi-Purpose Dataset of Webpages and Its Content Blocks: Design and Structure Validation. Appl. Sci., 11.10.3390/app11083319 ↗
23. (2023, March 28). Dataset-Popular 2014. A Dataset of Popular Pages (Taken from dir.yahoo.com) with Manually Marked up Semantic Blocks. Available online: https://github.com/rkrzr/dataset-popular.
24. Sanoja, A., and Gançarski, S. (2014, January 12–14). Block-o-matic: A web page segmentation framework. Proceedings of the 2014 International Conference on Multimedia Computing and Systems (ICMCS), Marrakesh, Morocco.10.1109/ICMCS.2014.6911249 ↗
25. Cai, D., Yu, S., Wen, J.-R., Ma, W.-Y., and Vips: A Vision-Based Page Segmentation Algorithm (2023, March 28). Microsoft Technical Report, MSR-TR-2003-79. Available online: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-2003-79.pdf.
26. Identifying semantic blocks in Web pages using Gestalt laws of grouping10.1007/s11280-015-0370-0 ↗
27. Constructing Novel Block Layouts for Webpage Analysis10.1145/3326457 ↗
28. A New Semantic Approach to Improve Webpage Segmentation
29. Debnath, S., Mitra, P., and Giles, C.L. (2005, January 25–28). Identifying content blocks from web documents. Proceedings of the Foundations of Intelligent Systems: 15th International Symposium, ISMIS 2005, Saratoga Springs, NY, USA.
30. Li, Y., and Yang, J. (2009, January 25–26). A novel method to extract informative blocks from web pages. Proceedings of the 2009 International Joint Conference on Artificial Intelligence, Hainan, China.10.1109/JCAI.2009.156 ↗
31. Grigera, J., Gardey, J.C., Rossi, G., and Garrido, A. Proceedings of the Web Information Systems and Technologies: 16th International Conference, WEBIST 2020, Virtual Events, 3–5 November 2020, and 17th International Conference, WEBIST 2021, 26–28 October 2021, Virtual Events, Springer International Publishing. Revised Selected Papers.
32. Information extraction meets the semantic web: A survey10.3233/SW-180333 ↗
33. Fauzi, F., Hong, J.L., and Belkhatir, M. (2009, January 19–25). Webpage segmentation for extracting images and their surrounding contextual information. Proceedings of the 17th ACM International Conference on Multimedia, Beijing, China.10.1145/1631272.1631379 ↗
34. Generalized hamming distance10.1023/A:1020499411651 ↗
35. Kiesel, J., Kneist, F., Meyer, L., Komlossy, K., Stein, B., and Potthast, M. (2020, January 19–23). Web page segmentation revisited: Evaluation framework and dataset. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Online.10.1145/3340531.3412782 ↗
36. Element similarity measures in XML schema matching10.1016/j.ins.2010.08.022 ↗
37. Andrew, J.J., Ferrari, S., Maurel, F., Dias, G., and Giguet, E. (2019, January 13–15). Web page segmentation for non visual skimming. Proceedings of the 33rd Pacific Asia Conference on Language, Information and Computation (PACLIC 33), Hakodate, Japan.
38. Multimodal Web Page Segmentation Using Self-organized Multi-objective Clustering10.1145/3480966 ↗
39. Extraction of Information from Web Page Using Content Mining Approach
40. FreeHand Sketch-based Authenticated Security System using Sequence Matcher
41. Wang, Q., Fang, Y., Ravula, A., Feng, F., Quan, X., and Liu, D. (2022, January 25–29). Webformer: The web-page transformer for structure information extraction. Proceedings of the ACM Web Conference 2022, Lyon, France.10.1145/3485447.3512032 ↗