STOD: Towards Scalable Task-Oriented Dialogue System on MultiWOZ-API
Source abstract
Abstract
Task-oriented dialogue systems (TODs) enable users to complete specific goals and are widely used in practice. Although existing models have achieved delightful performance for single-domain dialogues, scalability to new domains is far from well explored. Traditional dialogue systems rely on domain-specific information like dialogue state and database (DB), which limits the scalability of such systems. In this paper, we propose a Scalable Task-Oriented Dialogue modeling framework (STOD). Instead of labeling multiple dialogue components, which have been adopted by previous work, we only predict structured API queries to interact with DB and generate responses based on the complete DB results. Further, we construct a new API-schema-based TOD dataset MultiWOZ-API with API query and DB result annotation based on MultiWOZ 2.1. We then propose MSTOD and CSTOD for multi-domain and cross-domain TOD systems, respectively. We perform extensive qualitative experiments to verify the effectiveness of our proposed framework. We find the following. (1) Scalability across multiple domains: MSTOD achieves 2% improvements than the previous state-of-the-art in the multi-domain TOD. (2) Scalability to new domains: our framework enables satisfying generalization capability to new domains, a significant margin of 10% to existing baselines.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech and dialogue systems · Topic Modeling · Natural Language Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100083, China
Funding
National Natural Science Foundation of China
References (34)
Show source-provided references
1. Wen, T.-H., Gasic, M., Mrkšić, N., Su, P.-H., Vandyke, D., and Young, S. (2015, January 17–21). Semantically Conditioned LSTM-Based Natural Language Generation for Spoken Dialogue Systems. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.10.18653/v1/D15-1199 ↗
2. Eric, M., Goel, R., Paul, S., Sethi, A., Agarwal, S., Gao, S., Kumar, A., Goyal, A., Ku, P., and Hakkani-Tur, D. (2020, January 11–16). Multiwoz 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines. Proceedings of the 12th Language Resources and Evaluation Conference, Marseille, France.
3. Zhang, Z., Takanobu, R., Zhu, Q., Huang, M., and Zhu, X. (2020). Recent Advances and Challenges in Task-oriented Dialog System. arXiv.10.1007/s11431-020-1692-3 ↗
4. He, K., Lei, S., Yang, Y., Jiang, H., and Wang, Z. (2020, January 8–13). Syntactic Graph Convolutional Network for Spoken Language Understanding. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.10.18653/v1/2020.coling-main.246 ↗
5. Qin, L., Che, W., Li, Y., Wen, H., and Liu, T. (2019, January 3–7). A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Hong Kong, China.10.18653/v1/D19-1214 ↗
6. Xu, H., He, K., Yan, Y., Liu, S., Liu, Z., and Xu, W. (2020, January 13–18). A Deep Generative Distance-Based Classifier for Out-of-Domain Detection with Mahalanobis Space. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.10.18653/v1/2020.coling-main.125 ↗
7. Wu, C.-S., Madotto, A., Hosseini-Asl, E., Xiong, C., Socher, R., and Fung, P. (August, January 28). Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
8. Liu, S., Zhang, J., He, K., Xu, W., and Zhou, J. (2021). Scheduled Dialog Policy Learning: An Automatic Curriculum Learning Framework for Task-Oriented Dialog System. FINDINGS, Association for Computational Linguistics.10.18653/v1/2021.findings-acl.94 ↗
9. Su, S.-Y., Li, X., Gao, J., Liu, J., and Chen, Y.-N. (2018). Discriminative deep dyna-q: Robust planning for dialogue policy learning. arXiv.10.18653/v1/D18-1416 ↗
10. Peng, B., Zhu, C., Li, C., Li, X., Li, J., Zeng, M., and Gao, J. (2020). Few-shot natural language generation for task-oriented dialog. arXiv.10.18653/v1/2020.findings-emnlp.17 ↗
11. Lin, Z., Madotto, A., Winata, G.I., and Fung, P. (2020). Mintl: Minimalist transfer learning for task-oriented dialogue systems. arXiv.10.18653/v1/2020.emnlp-main.273 ↗
12. Soloist: Building task bots at scale with transfer learning and machine teaching10.1162/tacl_a_00399 ↗
13. Su, Y., Shu, L., Mansimov, E., Gupta, A., Cai, D., Lai, Y.-A., and Zhang, Y. (2022, January 22–27). Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Dublin, Ireland.10.18653/v1/2022.acl-long.319 ↗
14. Sun, H., Bao, J., Wu, Y., and He, X. (2022). BORT: Back and Denoising Reconstruction for End-to-End Task-Oriented Dialog. Findings of the Association for Computational Linguistics: NAACL 2022, Association for Computational Linguistics.10.18653/v1/2022.findings-naacl.166 ↗
15. Yang, Y., Ding, H., Liu, Q., and Quan, X. (2022). Ubarv2: Towards mitigating exposure bias in task-oriented dialogs. arXiv.
16. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019, January 2–7). Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the NAACL-HLT 2019, Minneapolis, MN, USA.
17. Language models are unsupervised multitask learners
18. He, K., Zhang, J., Yan, Y., Xu, W., Niu, C., and Zhou, J. (2020, January 8–13). Contrastive Zero-Shot Learning for Cross-Domain Slot Filling with Adversarial Attack. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.10.18653/v1/2020.coling-main.126 ↗
19. Wang, L., Li, X., Liu, J., He, K., Yan, Y., and Xu, W. (2021, January 7–11). Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-Explore Zero-Shot Learning for Slot Filling. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online.10.18653/v1/2021.emnlp-main.746 ↗
20. Chen, T., Kornblith, S., Norouzi, M., and Hinton, G.E. (2020). A simple framework for contrastive learning of visual representations. arXiv.
21. Lin, Z., Liu, B., Moon, A., Crook, P.A., Zhou, Z., Wang, Z., Yu, Z., Madotto, A., Cho, E., and Subba, R. (2021, January 6–11). Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue Statetracking. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.10.18653/v1/2021.naacl-main.448 ↗
22. Mo, K., Zhang, Y., Yang, Q., and Fung, P. (2018). Cross-domain dialogue policy transfer via simultaneous speech-act and slot alignment. arXiv.
23. Tickettalk: Toward Human-Level Performance with End-to-End, Transaction-Based Dialog Systems
24. Exploring the limits of transfer learning with a unified text-to-text transformer
25. Nekvinda, T., and Dušek, O. (2021, January 5–6). Shades of Bleu, Flavours of Success: The Case of Multiwoz. Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021), Bangkok, Thailand.10.18653/v1/2021.gem-1.4 ↗
26. Zhang, Y., Ou, Z., and Yu, Z. (2020, January 7–12). Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i05.6507 ↗
27. Kulhánek, J., Hudecek, V., Nekvinda, T., and Dušek, O. (2021). Augpt: Dialogue with pre-trained language models and data augmentation. arXiv.
28. Lin, Z., Madotto, A., Winata, G.I., and Fung, P.N. (2020, January 16–20). Mintl: Minimalist Transfer Learning for Task-Oriented Dialogue Systems. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.10.18653/v1/2020.emnlp-main.273 ↗
29. Yang, Y., Li, Y., and Quan, X. (2021, January 2–9). Ubar: Towards Fully End-to-End Task-Oriented Dialog System with gpt-2. Proceedings of the AAAI Conference on Artificial Intelligence, Virtually.10.1609/aaai.v35i16.17674 ↗
30. Lee, Y. (2021). Improving End-to-end Task-Oriented Dialog System with a Simple Auxiliary Task. Findings of the Association for Computational Linguistics: EMNLP 2021, Association for Computational Linguistics.10.18653/v1/2021.findings-emnlp.112 ↗
31. He, W., Dai, Y., Zheng, Y., Wu, Y., Cao, Z., Liu, D., Jiang, P., Yang, M., Huang, F., and Si, L. (March, January 22). Galaxy: A Generative Pre-Trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.
32. Ou, Z., Feng, J., Li, J., Li, Y., Liu, H., Peng, H., Huang, Y., and Zhao, J. (2022). A challenge on semi-supervised and reinforced task-oriented dialog systems. arXiv.
33. Peng, B., Li, X., Gao, J., Liu, J., Wong, K.-F., and Su, S.-Y. (2018, January 15–20). Deep Dyna-q: Integrating Planning for Task-Completion Dialogue Policy Learning. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia.10.18653/v1/P18-1203 ↗
34. Measuring nominal scale agreement among many raters10.1037/h0031619 ↗