A Topical Category-Aware Neural Text Summarizer
Source abstract
Abstract
The advent of the sequence-to-sequence model and the attention mechanism has increased the comprehension and readability of automatically generated summaries. However, most previous studies on text summarization have focused on generating or extracting sentences only from an original text, even though every text has a latent topic category. That is, even if a topic category helps improve the summarization quality, there have been no efforts to utilize such information in text summarization. Therefore, this paper proposes a novel topical category-aware neural text summarizer which is differentiated from legacy neural summarizers in that it reflects the topic category of an original text into generating a summary. The proposed summarizer adopts the class activation map (CAM) as topical influence of the words in the original text. Since the CAM excerpts the words relevant to a specific category from the text, it allows the attention mechanism to be influenced by the topic category. As a result, the proposed neural summarizer reflects the topical information of a text as well as the content information into a summary by combining the attention mechanism and CAM. The experiments on The New York Times Annotated Corpus show that the proposed model outperforms the legacy attention-based sequence-to-sequence model, which proves that it is effective at reflecting a topic category into automatic summarization.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Advanced Text Analysis Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Engineering, Kyung Hee University, Yongin 17104, Korea
References (34)
Show source-provided references
1. Cheng, J., and Lapata, M. (2016, January 7–12). Neural Summarization by Extracting Sentences and Words. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.10.18653/v1/P16-1046 ↗
2. Nallapati, R., Zhou, B., dos Santos, C., Gulcehre, C., and Xiang, B. (2016, January 11–12). Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond. Proceedings of the 20th Special Interest Group on Natural Language Learning Conference on Computational Natural Language Learning, Berlin, Germany.10.18653/v1/K16-1028 ↗
3. Yasunaga, M., Zhang, R., Meelu, K., Pareek, A., Srinivasan, K., and Radev, D. (2017, January 3–4). Graph-based Neural Multi-Document Summarization. Proceedings of the 21st Conference on Computational Natural Language Learning, Vancouver, BC, Canada.10.18653/v1/K17-1045 ↗
4. Narayan, S., Cohen, S., and Lapata, M. (2018, January 1–6). Ranking Sentences for Extractive Summarization with Reinforcement Learning. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, New Orleans, LA, USA.10.18653/v1/N18-1158 ↗
5. See, A., Liu, P.J., and Manning, C. (2017, January 30–4). Get To The Point: Summarization with Pointer-Generator Networks. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada.10.18653/v1/P17-1099 ↗
6. Paulus, R., Xiong, C., and Socher, R. (2017). A Deep Reinforced Model for Abstractive Summarization. arXiv.
7. Gehring, J., Auli, M., Grangier, D., Yarats, D., and Dauphin, Y. (2017, January 6–11). Convolutional Sequence to Sequence Learning. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia.
8. Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014, January 25–29). Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar.10.3115/v1/D14-1179 ↗
9. Rush, A., Chopra, S., and Weston, J. (2015, January 17–21). A Neural Attention Model for Abstractive Sentence Summarization. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.10.18653/v1/D15-1044 ↗
10. Nallapati, R., Zhai, F., and Zhou, B. (2017, January 4–9). Summarunner: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents. Proceedings of the 31th American Association for Artificial Intelligence Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.10958 ↗
11. Allamanis, M., Peng, H., and Sutton, C. (2016, January 20–22). A Convolutional Attention Network for Extreme Summarization of Source Code. Proceedings of the International Conference on Machine Learning, New York, NY, USA.
12. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2016, January 27–30). Learning Deep Features for Discriminative Localization. Proceedings of the Institute of Electrical and Electronics Engineers Conference on Computer Vision and Pattern Recognition, Las Vegas, NA, USA.10.1109/CVPR.2016.319 ↗
13. Bahdanau, D., Cho, K., and Bengio, Y. (2015, January 7–9). Neural Machine Translation by Jointly Learning to Align and Translate. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA.
14. Zhou, Q., Yang, N., Wei, F., Huang, S., Zhou, M., and Zhao, T. (2018, January 15–20). Neural Document Summarization by Jointly Learning to Score and Select Sentences. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia.10.18653/v1/P18-1061 ↗
15. Lexrank: Graph-based Lexical Centrality as Salience in Text Summarization10.1613/jair.1523 ↗
16. Sutskever, I., Vinyals, O., and Le, Q. (2014, January 8–13). Sequence to Sequence Learning with Neural Networks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
17. Cao, Z., Wei, F., Dong, L., Li, S., and Zhou, M. (2015, January 25–30). Ranking with Recursive Neural Networks and Its Application to Multi-Document Summarization. Proceedings of the 29th American Association for Artificial Intelligence Conference on Artificial Intelligence, Austin, TX, USA.10.1609/aaai.v29i1.9490 ↗
18. Chopra, S., Auli, M., and Rush, A. (2016, January 13–15). Abstractive Sentence Summarization with Attentive Recurrent Neural Networks. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics, San Diego, CA, USA.10.18653/v1/N16-1012 ↗
19. Vinyals, O., Fortunato, M., and Jaitly, N. (2015, January 7–12). Pointer Networks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
20. Tu, Z., Lu, Z., Liu, Y., Liu, X., and Li, H. (2016, January 7–12). Modeling Coverage for Neural Machine Translation. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.10.18653/v1/P16-1008 ↗
21. Krishna, K., and Srinivasan, B. (2018, January 1–6). Generating Topic-Oriented Summaries Using Neural Attention. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, New Orleans, LA, USA.10.18653/v1/N18-1153 ↗
22. Wang, L., Yao, J., Tao, Y., Zhong, L., Liu, W., and Du, Q. (2018, January 13–19). A Reinforced Topic-Aware Convolutional Sequence-to-Sequence model for Abstractive Text Summarization. Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden.10.24963/ijcai.2018/619 ↗
23. Li, X., Shen, Y., Du, L., and Xiong, C. (2010, January 26–30). Exploiting Novelty, Coverage and Balance for Topic-Focused Multi-Document Summarization. Proceedings of the 19th Association for Computing Machinery International Conference on Information and Knowledge Management, Toronto, ON, Canada.10.1145/1871437.1871724 ↗
24. Narayan, S., Cohen, S., and Lapata, M. (November, January 31). Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1206 ↗
25. Kim, Y. (2014, January 25–29). Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar.10.3115/v1/D14-1181 ↗
26. Oquab, M., Bottou, L., Laptev, I., and Sivic, J. (2015, January 7–12). Is Object Localization for Free? Weakly Supervised Learning with Convolutional Neural Networks. Proceedings of the Institute of Electrical and Electronics Engineers Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298668 ↗
27. Sandhaus, E. (2008). The New York Times Annotated Corpus, Linguistic Data Consortium.
28. Kingma, D., and Ba, J. (2015, January 7–9). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA.
29. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014, January 12–13). Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. Proceedings of the Neural Information Processing Systems 2014 Workshop on Deep Learning, Montreal, QC, Canada.
30. A Learning Algorithm for Continually Running Fully Recurrent Neural Networks10.1162/neco.1989.1.2.270 ↗
31. Dropout: A Simple Way to Prevent Neural Networks from Overfitting
32. Lin, C., and Hovy, E. (2002, January 11–12). Manual and Automatic Evaluation of Summaries. Proceedings of the Association for Computational Linguistics-02 Workshop on Automatic Summarization, Philadelphia, PA, USA.10.3115/1118162.1118168 ↗
33. Celikyilmaz, A., Bosselut, A., He, X., and Choi, Y. (2018, January 15–20). Deep Communicating Agents for Abstractive Summarization. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Melbourne, Australia.10.18653/v1/N18-1150 ↗
34. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is All You Need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.