Reliable Classification of FAQs with Spelling Errors Using an Encoder-Decoder Neural Network in Korean
Source abstract
Abstract
To resolve lexical disagreement problems between queries and frequently asked questions (FAQs), we propose a reliable sentence classification model based on an encoder-decoder neural network. The proposed model uses three types of word embeddings; fixed word embeddings for representing domain-independent meanings of words, fined-tuned word embeddings for representing domain-specific meanings of words, and character-level word embeddings for bridging lexical gaps caused by spelling errors. It also uses class embeddings to represent domain knowledge associated with each category. In the experiments with an FAQ dataset about online banking, the proposed embedding methods contributed to an improved performance of the sentence classification. In addition, the proposed model showed better performance (with an accuracy of 0.810 in the classification of 411 categories) than that of the comparison model.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Text and Document Classification Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Program of Computer and Communications Engineering, Kangwon National University, Chuncheon 24341, Korea
Funding
National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIP) · Shinban Bank
References (22)
Show source-provided references
1. Kalchbrenner, N., Grefenstette, E., and Blunsom, P. (2014, January 22–27). A Convolutional Neural Network for Modelling Sentences. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Baltimore, MD, USA.10.3115/v1/P14-1062 ↗
2. Kim, Y. (2014, January 25–29). Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar.10.3115/v1/D14-1181 ↗
3. Yin, W., and Schütze, H. (2015, January 30–31). Multichannel Variable-size Convolution for Sentence Classification. Proceedings of the Nineteenth Conference on Computational Natural Language Learning, Beijing, China.10.18653/v1/K15-1021 ↗
4. Yu, L., Hermann, K., Blunsom, K., and Pulman, S. (2014, January 12). Deep Learning for Answer Sentence Selection. Proceedings of the NIPS Deep Learning and Representation Learning Workshop, Montreal, QC, Canada.
5. Zhang, Y., Roller, S., and Byron, C. (2016, January 12–17). Mgnc-cnn: A Simple Approach to Exploiting Multiple Word Embeddings for Sentence Classification. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA.10.18653/v1/N16-1178 ↗
6. Mikolov, T., Sutskever, H., Chen, K., Corrado, G., and Dean, J. (2013, January 5–10). Distributed Representations of Words and Phrases and their Compositionality. Proceedings of the 26th International Conference on Neural Information Processing Systems (Volume 2), Lake Tahoe, NE, USA.
7. Hsu, S., Moon, C., Jones, P., and Nagiza, F. (2017, January 3–7). A Hybrid CNN-RNN Alignment Model for Phrase-Aware Sentence Classification. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, Valencia, Spain.10.18653/v1/E17-2071 ↗
8. Kim, Y., Jernite, Y., Sontag, D., and Rush, A. (2016, January 12–17). Character-Aware Neural Language Models. Proceedings of the AAAI 2016, Phoenix, AZ, USA.10.1609/aaai.v30i1.10362 ↗
9. Fully Character-Level Neural Machine Translation without Explicit Segmentation10.1162/tacl_a_00067 ↗
10. Ling, W., Trancoso, I., Dyer, C., and Black, A. (2016, January 7–12). Character-based Neural Machine Translation. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.
11. Park, S., Byun, J., Beak, S., Cho, Y., and Oh, A. (2018, January 15–20). Subword-level Word Vector Representations for Korean. Proceedings of the Association for Computational Linguistics, Melbourne, Australia.10.18653/v1/P18-1226 ↗
12. Seo, D., Chung, Y., and Kang, I. (2017, January 13–14). A typing error-robust Korean POS tagging using Hangul Jamo combination-based embedding. Proceedings of the of the HCLT, Daegu, Korea. (In Korean).
13. Zhang, X., Zhao, J., and LeCun, Y. (2015). Character-level Convolutional Networks for Text Classification. Advances in Neural Information Processing Systems 28(NIPS 2015), Courant Institute of Mathematical Sciences.
14. Kim, Y., Kim, D., Kumar, A., and Sarikaya, R. (2018, January 15–20). Efficient Large-Scale Domain Classification with Personalized Attention. Proceedings of the Association for Computational Linguistics, Melbourne, Australia.10.18653/v1/P18-1206 ↗
15. Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv.
16. Vaswabu, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
17. Chung, J., Gulcehre, C., Cho, K., and Bngio, Y. (2014). Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv.
18. Luong, M., Pham, H., and Manning, C. (2015, January 17–21). Effective Approaches to Attention-based Neural Machine Translation. Proceedings of the Association for Computational Linguistics, Lisbon, Portugal.10.18653/v1/D15-1166 ↗
19. Pennington, J., Socher, R., and Manning, C. (2014, January 25–29). Glove: Global Vectors for Word Representation. Proceedings of the Empirical Methods in Natural Language Processing, Doha, Qatar.10.3115/v1/D14-1162 ↗
20. Kingma, D., and Ba, L. (2017). Adam: A Method for Stochastic Optimization. arXiv.
21. Robertson, S., Walker, S., Jones, S., Beaulieu, M., and Gatford, M. (1994, January 2–4). Okapi at TREC-3. Proceedings of the TREC-3, Gaithersburg, MD, USA.10.6028/NIST.SP.500-225.city ↗
22. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., and Le, Q.V. (2019). XLNet: Generalized Autoregressive Pretraining for Language Understanding. arXiv.