An Abstract Summarization Method Combining Global Topics
Source abstract
Abstract
Existing abstractive summarization methods only focus on the correlation between the original words and the summary words, ignoring the topics’ influence on the summaries. To this end, an abstract summarization method combining global topic information, ACGT, is proposed. A topic information extractor, based on Latent Dirichlet Allocation, is constructed to extract key topic information from the original text, and an attention module is built to fuse key topic information with the original text representation. The summary is then generated by combining a pointer generation network and coverage mechanism. With evaluation metrics of ROUGE-1, ROUGE-2, and ROUGE-L, the experimental results of ACGT in the English dataset CNN/Daily Mail are 0.96%, 2.44%, and 1.03% higher than the baseline model, respectively. In the Chinese dataset, LCSTS, ACGT shows a higher performance than the baseline method by 1.19%, 1.03%, and 0.85%, respectively. Our results demonstrate that the performance of summaries is significantly correlated with the number of topics that are introduced. Case studies show that the introduction of topic information can improve both the coverage of original text topics and the fluency of summaries.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Advanced Text Analysis Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science and Engineering Chongqing, Chongqing University of Technology, Chongqing 400050, China
Funding
National Social Science Foundation of China · Action Plan for High Quality Development of Postgraduate Education of Chongqing University of Technology
References (36)
Show source-provided references
1. The automatic creation of literature abstracts10.1147/rd.22.0159 ↗
2. Yuan, D., Wang, L., Wu, Q., Meng, F., Ngan, N., and Xu, L. (2022). Language Bias-Driven Self-Knowledge Distillation with Generalization Uncertainty for Reducing Language Bias in Visual Question Answering. Appl. Sci., 12.10.3390/app12157588 ↗
3. Jwa, H., Oh, D., Park, K., Kang, J., and Lim, H. (2019). Exbake: Automatic fake news detection model based on bidirectional encoder representations from transformers (bert). Appl. Sci., 9.10.3390/app9194062 ↗
4. Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to sequence learning with neural networks. arXiv.
5. Rush, A.M., Chopra, S., and Weston, J. (2015). A neural attention model for abstractive sentence summarization. arXiv.10.18653/v1/D15-1044 ↗
6. See, A., Liu, P.J., and Manning, C.D. (2017). Get to the Point: Summarization with Pointer-Generator Networks. arXiv.10.18653/v1/P17-1099 ↗
7. Lin, C.Y., and Hovy, E. (June, January 27). Automatic evaluation of summaries using n-gram co-occurrence statistics. Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, Edmonton, AB, Canada.10.3115/1073445.1073465 ↗
8. Abdel-Salam, S., and Rafea, A. (2022). Performance Study on Extractive Text Summarization Using BERT Models. Information, 13.10.3390/info13020067 ↗
9. An unsupervised method for extractive multi-document summarization based on centroid approach and sentence embeddings10.1016/j.eswa.2020.114152 ↗
10. Document vector embedding based extractive text summarization system for Hindi and English text10.1007/s10489-021-02871-9 ↗
11. Nallapati, R., Zhou, B., dos Santos, C.N., Gulcehre, C., and Xiang, B. (2022). Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond. arXiv.
12. Gehrmann, S., Deng, Y., and Rush, A.M. (2018). Bottom-up abstractive summarization. arXiv.10.18653/v1/D18-1443 ↗
13. Celikyilmaz, A., Bosselut, A., He, X., and Choi, Y. (2018). Deep communicating agents for abstractive summarization. arXiv.10.18653/v1/N18-1150 ↗
14. Gulcehre, C., Ahn, S., Nallapati, R., Zhou, B., and Bengio, Y. (2022). Pointing the Unknown Words. arXiv.
15. Gu, J., Lu, Z., Li, H., and Li, V.O.K. (2016). Incorporating Copying Mechanism in Sequence-to-Sequence Learning. arXiv.10.18653/v1/P16-1154 ↗
16. Vinyals, O., Fortunato, M., and Jaitly, N. (2015). Pointer networks. Adv. Neural Inf. Process. Syst., 28, Available online: https://proceedings.neurips.cc/paper/2015/file/29921001f2f04bd3baee84a12e98098f-Paper.pdf.
17. Ruan, Q., Ostendorff, M., and Rehm, G. (2022). Histruct+: Improving extractive text summarization with hierarchical structure information. arXiv.10.18653/v1/2022.findings-acl.102 ↗
18. Mao, Z., Wu, C.H., Ni, A., Zhang, Y., Zhang, R., Yu, T., Deb, B., Zhu, C., Awadallah, A.H., and Radev, D. (2021). Dyle: Dynamic latent extraction for abstractive long-input summarization. arXiv.10.18653/v1/2022.acl-long.118 ↗
19. Li, J., Shang, J., and McAuley, J. (2022). UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Mining. arXiv.10.18653/v1/2022.acl-long.426 ↗
20. Bahrainian, S.A., Feucht, S., and Eickhoff, C. (2022). NEWTS: A Corpus for News Topic-Focused Summarization. arXiv.10.18653/v1/2022.findings-acl.42 ↗
21. Li, M., Lin, X.X., Chen, X., Chang, J., Zhang, Q., Wang, F., Wang, T., Liu, Z., Chu, W., and Zhao, D. (2022). Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation. arXiv.10.18653/v1/2022.acl-long.304 ↗
22. Latent Dirichlet allocation
23. Automatic Summarization Algorithm Based on the Combined Features of LDA
24. Multi-document automatic summarization algorithm based on important topics of LDA
25. Topic-aware long text automatic summarization algorithm
26. Guo, J.-F., Fei, Y.-X., Sun, W.-B., Xie, P.-P., and Zhang, J. (2022). A PGN-GAN Text Summarization Model Fusion Topic. J. Chin. Comput. Syst., 1–7.
27. Chou, Y.C., Kuo, C.J., Chen, T.T., Horng, G.J., Pai, M.Y., Wu, M.E., Lin, Y.C., Huang, M.H., Su, M.Y., and Chen, Y.C. (2019). Deep-learning-based defective bean inspection with GAN-structured automated labeled data augmentation in coffee industry. Appl. Sci., 9.10.3390/app9194166 ↗
28. Onah, D.F.O., Pang, E.L.L., and El-Haj, M. (2022). A Data-driven Latent Semantic Analysis for Automatic Text Summarization using LDA Topic Modelling. arXiv.10.1109/BigData55660.2022.10020259 ↗
29. An extractive text summarization approach using tagged-LDA based topic modeling10.1007/s11042-020-09549-3 ↗
30. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
31. Hermann, K.M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., and Blunsom, P. (2015). Teaching machines to read and comprehend. Adv. Neural Inf. Process. Syst., 28.
32. Hu, B., Chen, Q., and Zhu, F. (2015). Lcsts: A large scale chinese short text summarization dataset. arXiv.10.18653/v1/D15-1229 ↗
33. Lin, C.Y. (2022, August 15). Rouge: A Package for Automatic Evaluation of Summaries. Proceedings of the Workshop on Text Summarization of ACL, Available online: https://aclanthology.org/W04-1013.pdf.
34. Adaptive subgradient methods for online learning and stochastic optimization
35. Wasson, M. (2022, August 15). Using Leading Text for News Summaries: Evaluation Results and Implications for Commercial Summarization Applications[C]//COLING 1998 Volume 2: The 17th International Conference on Computational Linguistics. Available online: https://aclanthology.org/C98-2217.pdf.10.3115/980432.980791 ↗
36. Multi-task learning for abstractive text summarization with key information guide network10.1186/s13634-020-00674-7 ↗