TOMDS (Topic-Oriented Multi-Document Summarization): Enabling Personalized Customization of Multi-Document Summaries
Source abstract
Abstract
In a multi-document summarization task, if the user can decide on the summary topic, the generated summary can better align with the reader’s specific needs and preferences. This paper addresses the issue of overly general content generation by common multi-document summarization models and proposes a topic-oriented multi-document summarization (TOMDS) approach. The method is divided into two stages: extraction and abstraction. During the extractive stage, it primarily identifies and retrieves paragraphs relevant to the designated topic, subsequently sorting them based on their relevance to the topic and forming an initial subset of documents. In the abstractive stage, building upon the transformer architecture, the process includes two parts: encoding and decoding. In the encoding part, we integrated an external discourse parsing module that focuses on both micro-level within-paragraph semantic relationships and macro-level inter-paragraph connections, effectively combining these with the implicit relationships in the source document to produce more enriched semantic features. In the decoding part, we incorporated a topic-aware attention mechanism that dynamically zeroes in on information pertinent to the chosen topic, thus guiding the summary generation process more effectively. The proposed model was primarily evaluated using the standard text summary dataset WikiSum. The experimental results show that our model significantly enhanced the thematic relevance and flexibility of the summaries and improved the accuracy of grammatical and semantic comprehension in the generated summaries.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Semantic Web and Ontologies · Data Quality and Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer and Cyber Sciences, Communication University of China, Beijing 100024, China
Convergence Media Center, Communication University of China, Beijing 100024, China
State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China
Funding
National Key Research and Development Program of China
References (46)
Show source-provided references
1. (2023, October 20). IDC: Expect 175 Zettabytes of Data Worldwide by 2025. Networkworld. Available online: https://www.networkworld.com/article/3325397/idc-expect-175-zettabytes-of-data-worldwide-by-2025.html.
2. Aggarwal, C.C. (2018). Neural Networks and Deep Learning, Springer.10.1007/978-3-319-94463-0 ↗
3. Zopf, M. (2018, January 1). Auto-hmds: Automatic construction of a large heterogeneous multilingual multi-document summarization corpus. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. Available online: https://aclanthology.org/L18-1510.
4. Liu, P.J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, L., and Shazeer, N. (May, January 30). Generating wikipedia by summarizing long sequences. Proceedings of the 6th International Conference on Learning Representations (ICLR 2018), Vancouver, BC, Canada.
5. Fabbri, A.R., Li, I., She, T., Li, S., and Radev, D.R. (2019, January 4). Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.10.18653/v1/P19-1102 ↗
6. Radev, D.R. (2000, January 7–8). A common theory of information fusion from multiple text sources step one: Cross-document structure. Proceedings of the 1st SIGdial Workshop on Discourse and Dialogue, Hong Kong, China.10.3115/1117736.1117745 ↗
7. Antognini, D., and Faltings, B. (2019, January 20). Learning to create sentence semantic relation graphs for multi-document summarization. Proceedings of the 2nd Workshop on New Frontiers in Summarization, arXiv: Computation and Language, Association for Computational Linguistics, Hong Kong, China.10.18653/v1/D19-5404 ↗
8. Yin, Y., Song, L., Su, J., Zeng, J., Zhou, C., and Luo, J. (2019, January 16). Graph-based neural sentence ordering. Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China.10.24963/ijcai.2019/748 ↗
9. Li, W., Xiao, X., Liu, J., Wu, H., Wang, H., and Du, J. (2020, January 20). Leveraging Graph to Improve Abstractive Multi-Document Summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.555 ↗
10. Russell, S.J., and Norvig, P. (2010). Artificial Intelligence a Modern Approach, Prentice Hall.
11. Lexrank: Graph-based lexical centrality as salience in text summarization10.1613/jair.1523 ↗
12. Mihalcea, R., and Tarau, P. (2004, January 25–26). TextRank: Bringing order into text. Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, Barcelona, Spain.
13. Centroid-based summarization of multiple documents10.1016/j.ipm.2003.10.006 ↗
14. Liu, N., Peng, X., Lu, Y., Tang, X.J., and Wang, H.W. (2016, January 23–26). A Topic Approach to Sentence Ordering for Multi-document Summarization. Proceedings of the 2016 IEEE Trustcom/BigDataSE/ISPA, Tianjin, China.
15. Baralis, E., Cagliero, L., Jabeen, S., and Fiori, A. (2012, January 26–30). Multi-document summarization exploiting frequent itemsets. Proceedings of the 27th Annual ACM Symposium on Applied Computing, Trento, Italy.10.1145/2245276.2245427 ↗
16. Goldstein, J., Mittal, V.O., Carbonell, J.G., and Kantrowitz, M. (2000, January 30). Multi-document summarization by sentence extraction. Proceedings of the NAACL-ANLP 2000 Workshop: Automatic Summarization, Seattle, WA, USA.10.3115/1567564.1567569 ↗
17. Wan, X., and Yang, J. (2008, January 20–24). Multi-document summarization using cluster-based link analysis. Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Singapore.10.1145/1390334.1390386 ↗
18. Haghigh, A., and Vanderwende, L. (2009, January 4–9). Exploring content models for multi-document summarization. Proceedings of the Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, New York, NY, USA.10.3115/1620754.1620807 ↗
19. Arora, R., and Ravindran, B. (2008, January 15–19). Latent dirichlet allocation and singular value decomposition based multi-document summarization. Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, Pisa, Italy.10.1109/ICDM.2008.55 ↗
20. Jin, H., Wang, T., and Wan, X. (2020, January 5–10). Multi-granularity interaction network for extractive and abstractive multi-document summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.556 ↗
21. Sentence fusion for multidocument news summarization10.1162/089120105774321091 ↗
22. Tan, J., Wan, X., and Xiao, J. (August, January 30). Abstractive Document Summarization with a Graph-Based Attentional Neural Model. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada.
23. Liu, Y., and Lapata, M. (2019, January 30). Hierarchical Transformers for Multi-Document Summarization. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.10.18653/v1/P19-1500 ↗
24. Ma, C.B., Zhang, W.E., Wang, H., Gupta, S., and Guo, M. (2022, January 20–22). Incorporating linguistic knowledge for abstractive multi-document summarization. Proceedings of the 36th Pacific Asia Conference on Language, Information and Computation, Manila, Philippines.
25. Subtopic-based multi-modality ranking for topic-focused multi-document summarization10.1111/j.1467-8640.2012.00435.x ↗
26. A novel feature-based Bayesian model for query focused multi-document summarization10.1162/tacl_a_00212 ↗
27. Lee, S., Belkasim, S., and Zhang, Y. (2013, January 23–25). Multi-document text summarization using topic model and fuzzy logic. Proceedings of the 9th International Conference on Machine Learning and Data Mining in Pattern Recognition (MLDM 13), New York, NY, USA.10.1007/978-3-642-39712-7_12 ↗
28. Liu, N., Tang, X.J., Lu, Y., Li, M.X., Wang, H.W., and Xiao, P. (2014, January 13–15). Topic-sensitive multi-document summarization algorithm. Proceedings of the 2014 Sixth International Symposium on Parallel Architectures, Algorithms and Programming, Beijing, China.10.1109/PAAP.2014.22 ↗
29. Cui, P., and Hu, L. (2021, January 7–11). Topic-Guided Abstractive Multi-Document Summarization. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, Dominican Republic.10.18653/v1/2021.findings-emnlp.126 ↗
30. Chomsky, N. (2000). New Horizons in the Study of Language and Mind, Cambridge University Press.10.1017/CBO9780511811937 ↗
31. Chomsky, N. (2006). Language and Mind, Cambridge University Press.10.1017/CBO9780511791222 ↗
32. Xu, J. (2010). Text Linguistics in Contemporary Chinese, The Commercial Press.
33. Relational propositions in discourse10.1080/01638538609544632 ↗
34. Van Dijk, T.A. (2019). Macrostructures: An Interdisciplinary Study of Global Structures in Discourse, Interaction, and Cognition, Lawrence Erlbaum Associates, Inc.10.4324/9780429025532 ↗
35. Single Document Extractive Summarization with Satellite and Nuclear Relations
36. Research Automatics Summarization Coherence Based on Discourse Rhetoric Structure
37. Xu, J.C., Gan, Z., Cheng, Y., and Liu, J. (2020, January 5–10). Discourse-Aware Neural Extractive Text Summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.451 ↗
38. Attention is all you need
39. Zhang, X., Wei, Q., Song, Q., and Zhang, P. (2023, January 6–8). An Extractive Text Summarization Model Based on Rhetorical Structure Theory. Proceedings of the 26th ACIS International Winter Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD-Winter), Taiyuan, China.10.1109/SNPD-Winter57765.2023.10223980 ↗
40. Top-down RST parsing utilizing granularity levels in documents
41. Lin, C.Y. (2004, January 25–26). ROUGE: A Package for Automatic Evaluation of Summaries. Proceedings of the Workshop on Text Summarization Branches Out, Barcelona, Spain. Available online: http://anthology.aclweb.org/W/W04/W04-1013.pdf.
42. Kingma, D.P., and Ba, J. (2015, January 7–9). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference for Learning Representations, San Diego, CA, USA.
43. GraphSum: Discovering correlations among multiple terms for graph-based summarization10.1016/j.ins.2013.06.046 ↗
44. Discourse constraints for document compression10.1162/coli_a_00004 ↗
45. Narayan, S., Shay, B.C., and Lapata, M. (2018, January 1–6). Ranking sentences for extractive summarization with reinforcement learning. Proceedings of the NAACL 2018, New Orleans, USA.10.18653/v1/N18-1158 ↗
46. Xu, J., and Durrett, G. (2019, January 3–7). Neural Extractive Text Summarization with Syntactic Compression. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.10.18653/v1/D19-1324 ↗