A Deep Fusion Matching Network Semantic Reasoning Model
Source abstract
Abstract
As the vital technology of natural language understanding, sentence representation reasoning technology mainly focuses on sentence representation methods and reasoning models. Although the performance has been improved, there are still some problems, such as incomplete sentence semantic expression, lack of depth of reasoning model, and lack of interpretability of the reasoning process. Given the reasoning model’s lack of reasoning depth and interpretability, a deep fusion matching network is designed in this paper, which mainly includes a coding layer, matching layer, dependency convolution layer, information aggregation layer, and inference prediction layer. Based on a deep matching network, the matching layer is improved. Furthermore, the heuristic matching algorithm replaces the bidirectional long-short memory neural network to simplify the interactive fusion. As a result, it improves the reasoning depth and reduces the complexity of the model; the dependency convolution layer uses the tree-type convolution network to extract the sentence structure information along with the sentence dependency tree structure, which improves the interpretability of the reasoning process. Finally, the performance of the model is verified on several datasets. The results show that the reasoning effect of the model is better than that of the shallow reasoning model, and the accuracy rate on the SNLI test set reaches 89.0%. At the same time, the semantic correlation analysis results show that the dependency convolution layer is beneficial in improving the interpretability of the reasoning process.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Computational and Text Analysis Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Automation, University of Electronic Science and Technology of China, Chengdu 610054, China
Department of Geography and Anthropology, Louisiana State University, Baton Rouge, LA 70803, USA
Funding
Sichuan Science and Technology Program
References (38)
Show source-provided references
1. Convolutional neural network architectures for matching natural language sentences
2. Dynamic embedding projection-gated convolutional neural networks for text classification10.1109/TNNLS.2020.3036192 ↗
3. Using recurrent neural network structure with Enhanced Multi-Head Self-Attention for sentiment analysis10.1007/s11042-020-10336-3 ↗
4. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv.
5. MacCartney, B. (2009). Natural Language Inference, Stanford University.
6. A label-oriented loss function for learning sentence representations10.1016/j.csl.2020.101165 ↗
7. Moldovan, D., Clark, C., Harabagiu, S., and Maiorano, S. (June, January 27). Cogex: A logic prover for question answering. Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, Stroudsburg, PA, USA.10.3115/1073445.1073467 ↗
8. Raina, R., Ng, A.Y., and Manning, C.D. (2005, January 9–13). Robust textual inference via learning and abductive reasoning. Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI-05), Pittsburgh, PA, USA.
9. Le, N.Q.K., Yapp, E.K.Y., Nagasundaram, N., and Yeh, H.-Y. (2019). Classifying promoters by interpreting the hidden information of DNA sequences via deep learning and combination of continuous FastText N-grams. Front. Bioeng. Biotechnol., 305.10.3389/fbioe.2019.00305 ↗
10. A transformer architecture based on BERT and 2D convolutional neural network to identify DNA enhancers from sequence information10.1093/bib/bbab005 ↗
11. Ni, X., Yin, L., Chen, X., Liu, S., Yang, B., and Zheng, W. (2019). Semantic representation for visual reasoning. MATEC Web of Conferences, EDP Sciences.10.1051/matecconf/201927702006 ↗
12. Zhu, Y., Ko, T., Snyder, D., Mak, B., and Povey, D. (2018, January 2–6). Self-attentive speaker embeddings for text-independent speaker verification. Proceedings of the Interspeech 2018, 19th Annual Conference of the International Speech Communication Association, Hyderabad, India.10.21437/Interspeech.2018-1158 ↗
13. Williams, A., Nangia, N., and Bowman, S.R. (2017). A broad-coverage challenge corpus for sentence understanding through inference. arXiv.10.18653/v1/N18-1101 ↗
14. Liu, Y., Wan, Y., He, L., Peng, H., and Yu, P.S. (2021, January 2–9). KG-BART: Knowledge graph-augmented BART for generative commonsense reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, online.10.1609/aaai.v35i7.16796 ↗
15. SACNN: Self-attentive convolutional neural network model for natural language inference10.1145/3426884 ↗
16. Variational Explanation Generator: Generating Explanation for Natural Language Inference using Variational Auto-Encoder
17. Wu, H., and Huang, J. (2021, January 5–9). Relative Position Representation over Interaction Space for Natural Language Inference. Proceedings of the 2021 IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, China.10.1109/ICME51207.2021.9428375 ↗
18. Hierarchical taxonomy-aware and attentional graph capsule RCNNs for large-scale multi-label text classification10.1109/TKDE.2019.2959991 ↗
19. Google’s multilingual neural machine translation system: Enabling zero-shot translation10.1162/tacl_a_00065 ↗
20. Duan, C., Cui, L., Chen, X., Wei, F., Zhu, C., and Zhao, T. (2018, January 13–19). Attention-Fused Deep Matching Network for Natural Language Inference. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden.10.24963/ijcai.2018/561 ↗
21. Mou, L., Peng, H., Li, G., Xu, Y., Zhang, L., and Jin, Z. (2015). Discriminative neural sentence modeling by tree-based convolution. arXiv.10.18653/v1/D15-1279 ↗
22. Parikh, A.P., Täckström, O., Das, D., and Uszkoreit, J. (2016). A decomposable attention model for natural language inference. arXiv.10.18653/v1/D16-1244 ↗
23. Chen, Q., Zhu, X., Ling, Z., Wei, S., Jiang, H., and Inkpen, D. (2016). Enhanced LSTM for natural language inference. arXiv.10.18653/v1/P17-1152 ↗
24. Chen, D., and Manning, C.D. (2014, January 25–29). A fast and accurate dependency parser using neural networks. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1082 ↗
25. De Marneffe, M.-C., MacCartney, B., and Manning, C.D. (2006, January 22–28). Generating typed dependency parses from phrase structure parses. Proceedings of the Fifth International Conference on Language Resources and Evaluation, LREC 2006, Genoa, Italy.
26. Mou, L., Men, R., Li, G., Xu, Y., Zhang, L., Yan, R., and Jin, Z. (2015). Natural language inference by tree-based convolution and heuristic matching. arXiv.10.18653/v1/P16-2022 ↗
27. Graves, A., Jaitly, N., and Mohamed, A.-r. (2013, January 8–12). Hybrid speech recognition with deep bidirectional LSTM. Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic.10.1109/ASRU.2013.6707742 ↗
28. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst., 26.
29. Glorot, X., and Bengio, Y. (2010, January 13–15). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy.
30. Zhang, Z. (2018, January 4–6). Improved adam optimizer for deep neural networks. Proceedings of the 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS), Banff, AB, Canada.10.1109/IWQoS.2018.8624183 ↗
31. Bowman, S.R., Angeli, G., Potts, C., and Manning, C.D. (2015). A large annotated corpus for learning natural language inference. arXiv.10.18653/v1/D15-1075 ↗
32. Munkhdalai, T., and Yu, H. (August, January 30). Neural tree indexers for text understanding. Proceedings of the Conference. Association for Computational Linguistics. Meeting, Vancouver, BC, Canada.
33. Hybrid computing using a neural network with dynamic external memory10.1038/nature20101 ↗
34. Wang, S., and Jiang, J. (2015). Learning natural language inference with LSTM. arXiv.10.18653/v1/N16-1170 ↗
35. Sha, L., Chang, B., Sui, Z., and Li, S. (2016, January 11–16). Reading and thinking: Re-read lstm unit for textual entailment recognition. Proceedings of the COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan.
36. Liu, P., Qiu, X., Chen, J., and Huang, X.-J. (2016, January 7–12). Deep fusion lstms for text semantic matching. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.10.18653/v1/P16-1098 ↗
37. A Word Elimination Strategy for Learning Document Representation10.1088/1757-899X/466/1/012091 ↗
38. Bhowmik, P., Pantho, J.H., Mbongue, J.M., and Bobda, C. (2021, January 9–12). ESCA: Event-based split-CNN architecture with data-level parallelism on ultrascale+ FPGA. Proceedings of the 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), Orlando, FL, USA.10.1109/FCCM51124.2021.00028 ↗