Sentence Representation Method Based on Multi-Layer Semantic Network
Source abstract
Abstract
With the development of artificial intelligence, more and more people hope that computers can understand human language through natural language technology, learn to think like human beings, and finally replace human beings to complete the highly difficult tasks with cognitive ability. As the key technology of natural language understanding, sentence representation reasoning technology mainly focuses on the sentence representation method and the reasoning model. Although the performance has been improved, there are still some problems such as incomplete sentence semantic expression, lack of depth of reasoning model, and lack of interpretability of the reasoning process. In this paper, a multi-layer semantic representation network is designed for sentence representation. The multi-attention mechanism obtains the semantic information of different levels of a sentence. The word order information of the sentence is also integrated by adding the relative position mask between words to reduce the uncertainty caused by word order. Finally, the method is verified on the task of text implication recognition and emotion classification. The experimental results show that the multi-layer semantic representation network can promote sentence representation’s accuracy and comprehensiveness.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Technologies in Various Fields · Sentiment Analysis and Opinion Mining · Topic Modeling
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Automation, University of Electronic Science and Technology of China, Chengdu 610054, China
Department of Geography and Anthropology, Louisiana State University, Baton Rouge, LA 70803, USA
Funding
the Sichuan Science and Technology Program
References (42)
Show source-provided references
1. Natural-language generation—An overview10.1016/0020-7373(91)90032-3 ↗
2. Van Harmelen, F., Lifschitz, V., and Porter, B. (2008). Handbook of Knowledge Representation, Elsevier.
3. Macherey, K., Och, F.J., and Ney, H. (2001, January 3–7). Natural language understanding using statistical machine translation. Proceedings of the Seventh European Conference on Speech Communication and Technology, Aalborg, Denmark.10.21437/Eurospeech.2001-520 ↗
4. Natural language question answering: The view from here10.1017/S1351324901002807 ↗
5. An improved method for soft tissue modeling10.1016/j.bspc.2020.102367 ↗
6. The retrieved urban LST in Beijing based on TM, HJ-1B and MODIS10.1007/s13369-015-1957-6 ↗
7. Semantic representation for visual reasoning10.1051/matecconf/201927702006 ↗
8. Fractal dimension analysis for seismicity spatial and temporal distribution in the circum-Pacific seismic belt10.1007/s12040-018-1040-2 ↗
9. Earthquakes spatio–temporal distribution and fractal analysis in the Eurasian seismic belt10.1007/s12210-020-00871-4 ↗
10. Faruqui, M., and Dyer, C. (2014, January 26–30). Improving vector space word representations using multilingual correlation. Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, Gothenburg, Sweden.10.3115/v1/E14-1049 ↗
11. Skip-thought vectors
12. Construction of force haptic reappearance system based on Geomagic Touch haptic device10.1016/j.cmpb.2020.105344 ↗
13. Hill, F., Cho, K., and Korhonen, A. (2016, January 12–17). Learning distributed representations of sentences from unlabelled data. Proceedings of the 15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016, San Diego, CA, USA.10.18653/v1/N16-1162 ↗
14. Polajnar, T., Rimell, L., and Clark, S. (2015, January 18). An exploration of discourse-based sentence spaces for compositional distributional semantics. Proceedings of the First Workshop on Linking Computational Models of Lexical, Sentential and Discourse-level Semantics, Lisbon, Portugal.10.18653/v1/W15-2701 ↗
15. Convolutional neural network architectures for matching natural language sentences
16. Ding, Y., Tian, X., Yin, L., Chen, X., Liu, S., Yang, B., and Zheng, W. (2019, January 23–25). Multi-scale Relation Network for Few-Shot Learning Based on Meta-learning. Proceedings of the International Conference on Computer Vision Systems, Thessaloniki, Greece.10.1007/978-3-030-34995-0_31 ↗
17. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
18. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014, January 12). Empirical evaluation of gated recurrent neural networks on sequence modeling. Proceedings of the NIPS 2014 Workshop on Deep Learning, Montreal, QC, Canada.
19. Temporal evolution characteristics of PM2. 5 concentration based on continuous wavelet transform10.1016/j.scitotenv.2019.134244 ↗
20. Zhao, H., Lu, Z., and Poupart, P. (2015, January 25–31). Self-adaptive hierarchical sentence model. Proceedings of the 24th International Conference on Artificial Intelligence, Buenos Aires, Argentina.
21. Sun, C., Liu, Y., Liu, B., and Lin, L. (2017, January 7–10). Recognizing Text Entailment via Bidirectional LSTM Model with Inner-Attention. Proceedings of the International Conference on Intelligent Computing, Liverpool, UK.10.1007/978-3-319-63315-2_39 ↗
22. A Word Elimination Strategy for Learning Document Representation
23. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 6). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
24. Shen, T., Jiang, J., Zhou, T., Pan, S., Long, G., and Zhang, C. (2018, January 2–7). Disan: Directional self-attention network for RnN/CNN-free language understanding. Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11941 ↗
25. Schluter, N., and Varab, D. (November, January 31). When data permutations are pathological: The case of neural natural language inference. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1534 ↗
26. Bowman, S., Angeli, G., Potts, C., and Manning, C.D. (2015, January 17–21). A large annotated corpus for learning natural language inference. Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal.10.18653/v1/D15-1075 ↗
27. Bowman, S.R., Angeli, G., Potts, C., and Manning, C.D. (2020, December 20). The SNLI Corpus. Available online: http://hdl.handle.net/2451/41728.
28. Do, V., Camburu, O.-M., Akata, Z., and Lukasiewicz, T. (2020). e-SNLI-VE-2.0: Corrected Visual-Textual Entailment with Natural Language Explanations. arXiv.
29. Zhu, Y., Ko, T., Snyder, D., Mak, B., and Povey, D. (2018). Self-Attentive Speaker Embeddings for Text-Independent Speaker Verification. Proc. Interspeech, 3573–3577.10.21437/Interspeech.2018-1158 ↗
30. Williams, A., Nangia, N., and Bowman, S. (2018, January 1–6). A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, LA, USA. (Long Papers).10.18653/v1/N18-1101 ↗
31. Camburu, O.-M., Rocktäschel, T., Lukasiewicz, T., and Blunsom, P. (2018). e-snli: Natural language inference with natural language explanations. arXiv.
32. Recent advances in clinical natural language processing in support of semantic analysis
33. Wavelet analysis of the temporal-spatial distribution in the Eurasia seismic belt10.1142/S0219691317500187 ↗
34. Pennington, J., Socher, R., and Manning, C.D. (2014, January 25–29). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1162 ↗
35. Kim, Y. (2014, January 25–29). Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1181 ↗
36. Chen, Q., Ling, Z.-H., and Zhu, X. (2018, January 20–26). Enhancing Sentence Embedding with Generalized Pooling. Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, NM, USA.
37. End-to-end memory networks
38. Conneau, A., Kiela, D., Schwenk, H., Barrault, L., and Bordes, A. (2017, January 7). Supervised Learning of Universal Sentence Representations from Natural Language Inference Data. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.10.18653/v1/D17-1070 ↗
39. Dropout: A simple way to prevent neural networks from overfitting
40. Kingma, D.P., and Ba, J.L. (2015, January 7–9). Adam: A method for stochastic gradient descent. Proceedings of the ICLR: International Conference on Learning Representations, San Diego, CA, USA.
41. Bowman, S., Gupta, R., Gauthier, J., Manning, C.D., Rastogi, A., and Potts, C. (2016, January 7–12). A fast unified model for parsing and sentence understanding. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany.10.18653/v1/P16-1139 ↗
42. Graves, A., Jaitly, N., and Mohamed, A.-r. (2013, January 8–12). Hybrid speech recognition with deep bidirectional LSTM. Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic.10.1109/ASRU.2013.6707742 ↗