A Kazakh–Chinese Cross-Lingual Joint Modeling Method for Question Understanding
Source abstract
Abstract
Current research on intelligent question answering mainly focuses on high-resource languages such as Chinese and English, with limited studies on question understanding and reasoning in low-resource languages. In addition, during the joint modeling of question understanding tasks, the interdependence among subtasks can lead to error accumulation during the interaction phase, thereby affecting the prediction performance of the individual subtasks. To address the issue of error propagation caused by sentence-level intent encoding in the joint modeling of intent recognition and slot filling, this paper proposes a Cross-lingual Token-level Bi-Interactive Model (Bi-XTM). The model introduces a novel subtask interaction method that leverages the token-level intent output distribution as additional information for slot vector representation, effectively reducing error propagation and enhancing the information exchange between intent and slot vectors. Meanwhile, to address the scarcity of Kazakh (Arabic alphabet) language corpora, this paper constructs a cross-lingual joint question understanding dataset for the Xinjiang tourism domain, named JISD, which includes 16,548 Chinese samples and 1399 Kazakh samples. This dataset provides a new resource for cross-lingual intent recognition and slot filling joint tasks. Experimental results on the publicly available multi-lingual question understanding dataset MTOD and the newly constructed dataset demonstrate that the proposed Bi-XTM achieves state-of-the-art performance in both monolingual and cross-lingual settings.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Text and Document Classification Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Technology, Xinjiang Normal University, Urumqi 830054, China
School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China
The Base of Kazakh and Kirghiz Language of National Language Resource Monitoring and Research Center on Minority Languages, Urumqi 830017, China
Xinjiang Laboratory of Multi-Language Information Technology, Urumqi 830017, China
School of Big Data and Artificial Intelligence, Xinyang University, Xinyang 464000, China
Funding
National Natural Science Foundation of China
References (32)
Show source-provided references
1. Xu, P., and Sarikaya, R. (2013, January 8–12). Convolutional neural network based triangular crf for joint intent detection and slot filling. Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic.10.1109/ASRU.2013.6707709 ↗
2. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
3. Ravuri, S.V., and Stolcke, A. (2015, January 6–10). Recurrent neural network and LSTM models for lexical utterance classification. Proceedings of the 16th Annual Conference of the International Speech Communication Association, Interspeech, Dresden, Germany.10.21437/Interspeech.2015-42 ↗
4. Graves, A., and Graves, A. (2012). Long short-term memory. Supervised Sequence Labelling with Recurrent Neural Networks, Springer.10.1007/978-3-642-24797-2 ↗
5. Xia, C., Zhang, C., Yan, X., Chang, Y., and Yu, P.S. (2018). Zero-shot user intent detection via capsule neural networks. arXiv.10.18653/v1/D18-1348 ↗
6. Casanueva, I., Temčinas, T., Gerz, D., Henderson, M., and Vulić, I. (2020). Efficient intent detection with dual sentence encoders. arXiv.10.18653/v1/2020.nlp4convai-1.5 ↗
7. Yao, K., Zweig, G., Hwang, M.Y., Shi, Y., and Yu, D. (2013, January 25–29). Recurrent neural networks for language understanding. Proceedings of the 14th Annual Conference of the International Speech Communication Association, Interspeech, Lyon, France.10.21437/Interspeech.2013-569 ↗
8. Mesnil, G., He, X., Deng, L., and Bengio, Y. (2013, January 25–29). Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding. Proceedings of the 14th Annual Conference of the International Speech Communication Association, Interspeech, Lyon, France.10.21437/Interspeech.2013-596 ↗
9. Yao, K., Peng, B., Zhang, Y., Yu, D., Zweig, G., and Shi, Y. (2014, January 7–10). Spoken language understanding using long short-term memory neural networks. Proceedings of the 2014 IEEE Spoken Language Technology Workshop (SLT), South Lake Tahoe, CA, USA.10.1109/SLT.2014.7078572 ↗
10. Using recurrent neural networks for slot filling in spoken language understanding10.1109/TASLP.2014.2383614 ↗
11. Liu, B., and Lane, I. (2015, January 11). Recurrent neural network structured output prediction for spoken language understanding. Proceedings of the NIPS Workshop on Machine Learning for Spoken Language Understanding and Interactions, Montreal, QC, Canada.
12. Alam, M.T., Bhusal, D., Park, Y., and Rastogi, N. (2022). Cyner: A python library for cybersecurity named entity recognition. arXiv.
13. Dao, M.H., Truong, T.H., and Nguyen, D.Q. (2021). Intent detection and slot filling for Vietnamese. arXiv.10.21437/Interspeech.2021-618 ↗
14. Qin, L., Che, W., Li, Y., Wen, H., and Liu, T. (2019). A stack-propagation framework with token-level intent detection for spoken language understanding. arXiv.10.18653/v1/D19-1214 ↗
15. E, H., Niu, P., Chen, Z., and Song, M. (2019). A novel bi-directional interrelated model for joint intent detection and slot filling. arXiv.10.18653/v1/P19-1544 ↗
16. Zhang, C., Li, Y., Du, N., Fan, W., and Yu, P.S. (2018). Joint slot filling and intent detection via capsule neural networks. arXiv.10.18653/v1/P19-1519 ↗
17. Liu, Y., Meng, F., Zhang, J., Zhou, J., Chen, Y., and Xu, J. (2019). Cm-net: A novel collaborative memory network for spoken language understanding. arXiv.10.18653/v1/D19-1097 ↗
18. Qin, L., Liu, T., Che, W., Kang, B., Zhao, S., and Liu, T. (2021, January 6–11). A co-interactive transformer for joint slot filling and intent detection. Proceedings of the ICASSP 2021—2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), virtual.10.1109/ICASSP39728.2021.9414110 ↗
19. Liu, Z., Winata, G.I., Lin, Z., Xu, P., and Fung, P. (2020, January 7–12). Attention-informed mixed-language training for zero-shot cross-lingual task-oriented dialogue systems. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i05.6362 ↗
20. Xu, W., Haider, B., and Mansour, S. (2020). End-to-end slot alignment and recognition for cross-lingual NLU. arXiv.10.18653/v1/2020.emnlp-main.410 ↗
21. Qin, L., Ni, M., Zhang, Y., and Che, W. (2020). Cosda-ml: Multi-lingual code-switching data augmentation for zero-shot cross-lingual nlp. arXiv.10.24963/ijcai.2020/533 ↗
22. Bapna, A., Tur, G., Hakkani-Tur, D., and Heck, L. (2017). Towards zero-shot frame semantic parsing for domain scaling. arXiv.10.21437/Interspeech.2017-518 ↗
23. Liu, Z., Winata, G.I., Xu, P., and Fung, P. (2020). Coach: A coarse-to-fine approach for cross-domain slot filling. arXiv.10.18653/v1/2020.acl-main.3 ↗
24. Shah, D.J., Gupta, R., Fayazi, A.A., and Hakkani-Tur, D. (2019). Robust zero-shot cross-domain slot filling with example values. arXiv.10.18653/v1/P19-1547 ↗
25. Zeng, F., and Gao, W. (2023). Prompt to be consistent is better than self-consistent? few-shot and zero-shot fact verification with pre-trained language models. arXiv.10.18653/v1/2023.findings-acl.278 ↗
26. Lafferty, J., McCallum, A., and Pereira, F. (July, January 28). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proceedings of the ICML’01 Eighteenth International Conference on Machine Learning, Williamstown, MA, USA.
27. Coucke, A., Saade, A., Ball, A., Bluche, T., Caulier, A., Leroy, D., Doumouro, C., Gisselbrecht, T., Caltagirone, F., and Lavril, T. (2018). Snips voice platform: An embedded spoken language understanding system for private-by-design voice interfaces. arXiv.
28. Hemphill, C.T., Godfrey, J.J., and Doddington, G.R. (1990, January 24–27). The ATIS spoken language systems pilot corpus. Proceedings of the Speech and Natural Language: Proceedings of a Workshop, Hidden Valley, PA, USA.10.3115/116580.116613 ↗
29. Schuster, S., Gupta, S., Shah, R., and Lewis, M. (2018). Cross-lingual transfer learning for multilingual task oriented dialog. arXiv.10.18653/v1/N19-1380 ↗
30. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
31. Sang, E.F., and De Meulder, F. (2003). Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. arXiv.
32. Goo, C.W., Gao, G., Hsu, Y.K., Huo, C.L., Chen, T.C., Hsu, K.W., and Chen, Y.N. (2018, January 1–6). Slot-gated modeling for joint slot filling and intent prediction. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, LA, USA.10.18653/v1/N18-2118 ↗