Abstract Text Summarization with a Convolutional Seq2seq Model
Source abstract
Abstract
Abstract text summarization aims to offer a highly condensed and valuable information that expresses the main ideas of the text. Most previous researches focus on extractive models. In this work, we put forward a new generative model based on convolutional seq2seq architecture. A hierarchical CNN framework is much more efficient than the conventional RNN seq2seq models. We also equip our model with a copying mechanism to deal with the rare or unseen words. Additionally, we incorporate a hierarchical attention mechanism to model the keywords and key sentences simultaneously. Finally we verify our model on two real-life datasets, GigaWord and DUC corpus. The experiment results verify the effectiveness of our model as it outperforms state-of-the-art alternatives consistently and statistical significantly.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Advanced Text Analysis Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Science and Technology on Information Systems Engineering Laboratory, National University of Defense Technology, Changsha 410073, China
Funding
National Nature Science Foundation of China (NSFC)
References (37)
Show source-provided references
1. World Wide Web site summarization
2. Berend, G. (2011, January 8–13). Opinion Expression Mining by Exploiting Keyphrase Extraction. Proceedings of the Fifth International Joint Conference on Natural Language Processing, IJCNLP 2011, Chiang Mai, Thailand.
3. Bahdanau, D., Chorowski, J., Serdyuk, D., Brakel, P., and Bengio, Y. (2016, January 20–25). End-to-end attention-based large vocabulary speech recognition. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016, Shanghai, China.10.1109/ICASSP.2016.7472618 ↗
4. Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R.J., Darrell, T., and Saenko, K. (2015, January 7–13). Sequence to Sequence—Video to Text. Proceedings of the 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile.10.1109/ICCV.2015.515 ↗
5. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv.
6. Nallapati, R., Zhou, B., dos Santos, C.N., Gülçehre, Ç., and Xiang, B. (2016, January 11–12). Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond. Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, Berlin, Germany.10.18653/v1/K16-1028 ↗
7. Gehring, J., Auli, M., Grangier, D., Yarats, D., and Dauphin, Y.N. (2017, January 6–11). Convolutional Sequence to Sequence Learning. Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, Australia.
8. Zhang, Y., Fang, Y., and Xiao, W. (2017, January 11–13). Deep keyphrase generation with a convolutional sequence to sequence model. Proceedings of the 4th International Conference on Systems and Informatics, ICSAI 2017, Hangzhou, China.10.1109/ICSAI.2017.8248519 ↗
9. Dauphin, Y.N., Fan, A., Auli, M., and Grangier, D. (2017, January 6–11). Language Modeling with Gated Convolutional Networks. Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, Australia.
10. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
11. Gu, J., Lu, Z., Li, H., and Li, V.O.K. (2016, January 7–12). Incorporating Copying Mechanism in Sequence-to-Sequence Learning. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany. Long Papers.10.18653/v1/P16-1154 ↗
12. Neto, J.L., Freitas, A.A., and Kaestner, C.A.A. (2002, January 11–14). Automatic Text Summarization Using a Machine Learning Approach. Proceedings of the 16th Brazilian Symposium on Artificial Intelligence, SBIA 2002, Porto de Galinhas/Recife, Brazil.
13. Colmenares, C.A., Litvak, M., Mantrach, A., and Silvestri, F. (June, January 31). HEADS: Headline Generation as Sequence Prediction Using an Abstract Feature-Rich Space. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2015, The Denver, CO, USA.10.3115/v1/N15-1014 ↗
14. Long story short - Global unsupervised models for keyphrase based meeting summarization10.1016/j.specom.2010.06.002 ↗
15. Ribeiro, R., Marujo, L., de Matos, D.M., Neto, J.P., Gershman, A., and Carbonell, J.G. (August, January 28). Self reinforcement for important passage retrieval. Proceedings of the 36th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR ’13, Dublin, Ireland.
16. David Zajic, B.J.D., and Schwartz, R. (2004, January 2–7). Bbn/umd at duc-2004: Topiary. Proceedings of the North American Chapter of the Association for Computational Linguistics Workshop on Document Understanding, Boston, MA, USA.
17. Banko, M., Mittal, V.O., and Witbrock, M.J. (2000, January 1–8). Headline Generation Based on Statistical Translation. Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, Hong Kong, China.10.3115/1075218.1075259 ↗
18. Cohn, T., and Lapata, M. (2008, January 18–22). Sentence Compression Beyond Word Deletion. Proceedings of the 22nd International Conference on Computational Linguistics, Proceedings of the Conference, COLING 2008, Manchester, UK.10.3115/1599081.1599099 ↗
19. Woodsend, K., Feng, Y., and Lapata, M. (2010, January 9–11). Title Generation with Quasi-Synchronous Grammar. Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, EMNLP 2010, Cambridge, MA, USA.
20. Natural Language Processing (Almost) from Scratch
21. Rush, A.M., Chopra, S., and Weston, J. (2015, January 17–21). A Neural Attention Model for Abstractive Sentence Summarization. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal.10.18653/v1/D15-1044 ↗
22. Chopra, S., Auli, M., and Rush, A.M. (2016, January 12–17). Abstractive Sentence Summarization with Attentive Recurrent Neural Networks. Proceedings of the The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016, San Diego, CA, USA.10.18653/v1/N16-1012 ↗
23. Sankaran, B., Mi, H., Al-Onaizan, Y., and Ittycheriah, A. (2016). Temporal Attention Model for Neural Machine Translation. arXiv.
24. Hu, B., Chen, Q., and Zhu, F. (2015, January 17–21). LCSTS: A Large Scale Chinese Short Text Summarization Dataset. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal.10.18653/v1/D15-1229 ↗
25. Nallapati, R., Xiang, B., and Zhou, B. (2016). Sequence-to-Sequence RNNs for Text Summarization. arXiv.
26. Rekabdar, B., Mousas, C., and Gupta, B. (February, January 30). Generative Adversarial Network with Policy Gradient for Text Summarization. Proceedings of the 13th IEEE International Conference on Semantic Computing, ICSC 2019, Newport Beach, CA, USA.10.1109/ICOSC.2019.8665583 ↗
27. Zhang, H., Gong, Y., Yan, Y., Duan, N., Xu, J., Wang, J., Gong, M., and Zhou, M. (2019). Pretraining-Based Natural Language Generation for Text Summarization. arXiv.10.18653/v1/K19-1074 ↗
28. COSUM: Text summarization based on clustering and optimization10.1111/exsy.12340 ↗
29. Abstractive text summarization using LSTM-CNN based deep learning10.1007/s11042-018-5749-3 ↗
30. A text summarization method based on fuzzy rules and applicable to automated assessment10.1016/j.eswa.2018.07.047 ↗
31. Sutskever, I., Vinyals, O., and Le, Q.V. (2014, January 8–13). Sequence to Sequence Learning with Neural Networks. Proceedings of the Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, QC, Canada.
32. Cho, K., van Merrienboer, B., Gülçehre, Ç., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014, January 25–29). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, Doha, Qatar.10.3115/v1/D14-1179 ↗
33. Luong, T., Pham, H., and Manning, C.D. (2015, January 17–21). Effective Approaches to Attention-based Neural Machine Translation. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal.10.18653/v1/D15-1166 ↗
34. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
35. van den Oord, A., Kalchbrenner, N., and Kavukcuoglu, K. (2016, January 19–24). Pixel Recurrent Neural Networks. Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA.
36. Learning Motion Features for Example-Based Finger Motion Estimation for Virtual Characters10.1007/s13319-017-0136-9 ↗
37. Pascanu, R., Mikolov, T., and Bengio, Y. (2013, January 16–21). On the difficulty of training recurrent neural networks. Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA.