The Impact of Semi-Supervised Learning on the Performance of Intelligent Chatbot System
Source abstract
Abstract
Artificial intelligent based dialog systems are getting attention from both business and academic communities. The key parts for such intelligent chatbot systems are domain classification, intent detection, and named entity recognition. Various supervised, unsupervised, and hybrid approaches are used to detect each field. Such intelligent systems, also called natural language understanding systems analyze user requests in sequential order: domain classification, intent, and entity recognition based on the semantic rules of the classified domain. This sequential approach propagates the downstream error; i.e., if the domain classification model fails to classify the domain, intent and entity recognition fail. Furthermore, training such intelligent system necessitates a large number of user-annotated datasets for each domain. This study proposes a single joint predictive deep neural network framework based on long short-term memory using only a small user-annotated dataset to address these issues. It investigates value added by incorporating unlabeled data from user chatting logs into multi-domain spoken language understanding systems. Systematic experimental analysis of the proposed joint frameworks, along with the semi-supervised multi-domain model, using open-source annotated and unannotated utterances shows robust improvement in the predictive performance of the proposed multi-domain intelligent chatbot over a base joint model and joint model based on adversarial learning.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Sentiment Analysis and Opinion Mining · AI in Service Interactions
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (52)
Show source-provided references
1. ELIZA—a computer program for the study of natural language communication between man and machine10.1145/365153.365168 ↗
2. Intelligent personal assistants: A systematic literature review10.1016/j.eswa.2020.113193 ↗
3. Onenet: Joint domain, intent, slot prediction for spoken language understanding
4. Multi-domain joint semantic frame parsing using bi-directional RNN-LSTM
5. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
6. T. Miyato, A. M. Dai and I. Goodfellow, “Adversarial training methods for semi-supervised text classification”, arXiv preprint arXiv: 1605.07725, 2016.
7. Wallace · The elements of AIML style · 2016
8. Multi parameter design in AIML framework for balinese calendar knowledge access
9. A review of current trends in the development of chatbot systems
10. Survey on intelligent chatbots: State-of-the-art and future research directions
11. Survey of various AI chatbots based on technology used
12. Survey on chatbot design techniques in speech conversation systems
13. A study on conversational public administration service of the chatbot based on artificial intelligence
14. E. Okur, S. H. Kumar, S. Sahay, A. A. Esme and L. Nachman, “Natural language interactions in autonomous vehicles: Intent detection and slot filling from passenger utterances,” ArXiv Preprint ArXiv: 1904.10500, 2019.
15. Review of chatbots design techniques10.5120/ijca2018917606 ↗
16. Chatbot design method using hybrid word vector expression model based on real telemarketing data
17. Syntactic structured framework for resolving reflexive anaphora in Urdu discourse using multilingual NLP
18. Jumping NLP curves: A review of natural language processing research10.1109/MCI.2014.2307227 ↗
19. Towards end-to-end spoken language understanding
20. Application of deep belief networks for natural language understanding10.1109/TASLP.2014.2303296 ↗
21. Open intent extraction from natural language interactions
22. S. Schuster, S. Gupta, R. Shah and M. Lewis, “Cross-lingual transfer learning for multilingual task oriented dialog,” arXiv preprint arXiv: 1810.13327, 2018.10.18653/v1/N19-1380 ↗
23. S. Larson, A. Mahendran, J. J. Peper, C. Clarke, A. Lee et al., “An evaluation dataset for intent classification and out-of-scope prediction,” arXiv preprint arXiv: 1909.02027, 2019.10.18653/v1/D19-1131 ↗
24. Attention-based bidirectional long short-term memory networks for relation classification
25. Exploiting query click logs for utterance domain detection in spoken language understanding
26. Out-of-domain detection for natural language understanding in dialog systems10.1109/TASLP.2020.2983593 ↗
27. Contextual domain classification in spoken language understanding systems using recurrent neural network
28. R. Gupta, A. Rastogi and D. Hakkani-Tur, “An efficient approach to encoding context for spoken language understanding,” ArXiv Preprint ArXiv: 1807.00267, 2018.10.21437/Interspeech.2018-2403 ↗
29. B. Liu and I. Lane, “Attention-based recurrent neural network models for joint intent detection and slot filling,” ArXiv Preprint ArXiv: 1609.01454, 2016.10.21437/Interspeech.2016-1352 ↗
30. M. Ma, K. Zhao, L. Huang, B. Xiang and B. Zhou, “Jointly trained sequential labeling and classification by sparse attention neural networks,” ArXiv Preprint ArXiv: 1709.10191, 2017.10.21437/Interspeech.2017-1321 ↗
31. Slot-gated modeling for joint slot filling and intent prediction
32. P. G. Shivakumar, M. Yang and P. Georgiou, “Spoken language intent detection using confusion2vec,” ArXiv Preprint ArXiv: 1904.03576, 2019.10.21437/Interspeech.2019-2226 ↗
33. Multi-turn QA: A RNN contextual approach to intent classification for goal-oriented systems
34. Easy contextual intent prediction and slot detection
35. Y. Ji, C. Tan, S. Martschat, Y. Choi and N. A. Smith, “Dynamic entity representations in neural language models,” arXiv preprint arXiv: 1708.00781, 2017.10.18653/v1/D17-1195 ↗
36. N. T. Vu, “Sequential convolutional neural networks for slot filling in spoken language understanding,” ArXiv Preprint ArXiv: 1606.07783, 2016.
37. F. Dernoncourt, J. Y. Lee and P. Szolovits, “NeuroNER: An easy-to-use program for named-entity recognition based on neural networks,” ArXiv Preprint ArXiv: 1705.05487, 2017.10.18653/v1/D17-2017 ↗
38. M. Surdeanu, S. Gupta, J. Bauer, D. McClosky, A. X. Chang et al., “Standford's distantly-supervised slot-filling system,” 2011. [Online] Available: https://www.semanticscholar.org/paper/Stanford's-Distantly-Supervised-Slot-Filling-System-Surdeanu-Gupta/677455e832f1f07d060188238c4164e2450c3cd1.
39. Named entity recognition in tweets: An experimental study
40. Design challenges and misconceptions in named entity recognition
41. Improving slot filling in spoken language understanding with joint pointer and attention
42. G. Bekoulis, J. Deleu, T. Demeester and C. Develder, “Adversarial training for multi-context joint entity and relation extraction,” ArXiv Preprint ArXiv: 1808.06876, 2018.10.18653/v1/D18-1307 ↗
43. C. Zhang, Y. Li, N. Du, W. Fan and P. S. Yu, “Joint slot filling and intent detection via capsule neural networks,” ArXiv Preprint ArXiv: 1812.09471, 2018.10.18653/v1/P19-1519 ↗
44. S. Mehri, M. Eric and D. Hakkani-Tur, “Dialogue: A natural language understanding benchmark for task-oriented dialogue,” ArXiv Preprint ArXiv: 2009.13570, 2020.
45. M. Koziński, L. Simon and F. Jurie, “An adversarial regularisation for semi-supervised training of structured output neural networks,” ArXiv Preprint ArXiv: 1702.02382, 2017.
46. Semi-supervised training using adversarial multi-task learning for spoken language understanding
47. A semi-supervised approach for natural language call routing
48. S. Zhu, R. Cao and K. Yu, “Dual learning for semi-supervised natural language understanding,” ArXiv Preprint ArXiv: 2004.12299, 2020.10.1109/TASLP.2020.3001684 ↗
49. Distributed representations of words and phrases and their compositionality
50. Text classification using long short-term memory
51. Adversarial training for multi domain dialog system10.32604/iasc.2022.018757 ↗
52. X. Liu, A. Eshghi, P. Swietojanski and V. Rieser, “Benchmarking natural language understanding services for building conversational agents,” ArXiv Preprint ArXiv, 1903.05566, 2019.