Relation-Aware Graph Transformer for SQL-to-Text Generation
Source abstract
Abstract
Generating natural language descriptions for structured representation (e.g., a graph) is an important yet challenging task. In this work, we focus on SQL-to-text, a task that maps a SQL query into the corresponding natural language question. Previous work represents SQL as a sparse graph and utilizes a graph-to-sequence model to generate questions, where each node can only communicate with k-hop nodes. Such a model will degenerate when adapted to more complex SQL queries due to the inability to capture long-term and the lack of SQL-specific relations. To tackle this problem, we propose a relation-aware graph transformer (RGT) to consider both the SQL structure and various relations simultaneously. Specifically, an abstract SQL syntax tree is constructed for each SQL to provide the underlying relations. We also customized self-attention and cross-attention strategies to encode the relations in the SQL tree. Experiments on benchmarks WikiSQL and Spider demonstrate that our approach yields improvements over strong baselines.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Advanced Graph Neural Networks · Natural Language Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
X-LANCE Lab, MoE Key Lab of Artificial Intelligence, AI Institute, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
References (51)
Show source-provided references
1. Learning executable semantic parsers for natural language understanding10.1145/2866568 ↗
2. Sorokin, D., and Gurevych, I. (2018). Modeling semantics with gated graph neural networks for knowledge base question answering. arXiv.
3. Livowsky, J.M. (2003). Natural Language Interface for Searching Database. (6,598,039), U.S. Patent.
4. Shwartz, S., Fratarcangeli, C., Cullingford, R.E., Aimi, G.S., and Strasburger, D.P. (1993). Database Retrieval System Having a Natural Language Interface. (5,197,005), U.S. Patent.
5. Natural language interface for database: A brief review
6. Guo, D., Sun, Y., Tang, D., Duan, N., Yin, J., Chi, H., Cao, J., Chen, P., and Zhou, M. (November, January 31). Question Generation from SQL Queries Improves Neural Semantic Parsing. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1188 ↗
7. Wu, K., Wang, L., Li, Z., Zhang, A., Xiao, X., Wu, H., Zhang, M., and Wang, H. (2021, January 7–11). Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsing. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic.10.18653/v1/2021.emnlp-main.707 ↗
8. Cao, R., Chen, L., Chen, Z., Zhao, Y., Zhu, S., and Yu, K. (2021, January 2–4). LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relations. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Online.10.18653/v1/2021.acl-long.198 ↗
9. Chen, Z., Chen, L., Li, H., Cao, R., Ma, D., Wu, M., and Yu, K. (2021). Decoupled Dialogue Modeling and Semantic Parsing for Multi-Turn Text-to-SQL. arXiv.10.18653/v1/2021.findings-acl.270 ↗
10. Cao, R., Zhu, S., Liu, C., Li, J., and Yu, K. (August, January 28). Semantic Parsing with Dual Learning. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
11. Chen, Z., Chen, L., Zhao, Y., Cao, R., Xu, Z., Zhu, S., and Yu, K. (2021, January 6–11). ShadowGNN: Graph Projection Neural Network for Text-to-SQL Parser. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.10.18653/v1/2021.naacl-main.441 ↗
12. Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems, Curran Associates, Inc.
13. Iyer, S., Konstas, I., Cheung, A., and Zettlemoyer, L. (2016, January 7–12). Summarizing source code using a neural attention model. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.10.18653/v1/P16-1195 ↗
14. Xu, K., Wu, L., Wang, Z., Feng, Y., Witbrock, M., and Sheinin, V. (2018). Graph2seq: Graph to sequence learning with attention-based neural networks. arXiv.
15. Xu, K., Wu, L., Wang, Z., Feng, Y., and Sheinin, V. (2018, January 2–4). SQL-to-Text Generation with Graph-to-Sequence Model. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1112 ↗
16. The graph neural network model10.1109/TNN.2008.2005605 ↗
17. Zhong, V., Xiong, C., and Socher, R. (2017). Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv.
18. Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., and Roman, S. (2018). Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv.10.18653/v1/D18-1425 ↗
19. Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation10.1613/jair.5477 ↗
20. Gardent, C., Shimorina, A., Narayan, S., and Perez-Beltrachini, L. (2017, January 4–7). The WebNLG Challenge: Generating Text from RDF Data. Proceedings of the 10th International Conference on Natural Language Generation, Santiago de Compostela, Spain.10.18653/v1/W17-3518 ↗
21. Novikova, J., Dušek, O., and Rieser, V. (2017, January 15–17). The E2E Dataset: New Challenges For End-to-End Generation. Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, Saarbrücken, Germany.10.18653/v1/W17-5525 ↗
22. Banarescu, L., Bonial, C., Cai, S., Georgescu, M., Griffitt, K., Hermjakob, U., Knight, K., Koehn, P., Palmer, M., and Schneider, N. (2013, January 8–9). Abstract Meaning Representation for Sembanking. Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, Sofia, Bulgaria.
23. Konstas, I., Iyer, S., Yatskar, M., Choi, Y., and Zettlemoyer, L. (August, January 30). Neural AMR: Sequence-to-Sequence Models for Parsing and Generation. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, BC, Canada.
24. Castro Ferreira, T., Calixto, I., Wubben, S., and Krahmer, E. (2017, January 4–7). Linguistic realisation as machine translation: Comparing different MT models for AMR-to-text generation. Proceedings of the 10th International Conference on Natural Language Generation, Santiago de Compostela, Spain.10.18653/v1/W17-3501 ↗
25. Marcheggiani, D., and Perez-Beltrachini, L. (2018, January 5–8). Deep Graph Convolutional Encoders for Structured Data to Text Generation. Proceedings of the 11th International Conference on Natural Language Generation, Tilburg, The Netherlands.10.18653/v1/W18-6501 ↗
26. Beck, D., Haffari, G., and Cohn, T. (2018, January 16–18). Graph-to-Sequence Learning using Gated Graph Neural Networks. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia.10.18653/v1/P18-1026 ↗
27. Kipf, T.N., and Welling, M. (2017, January 24–26). Semi-Supervised Classification with Graph Convolutional Networks. Proceedings of the 5th International Conference on Learning Representations, ICLR ’17, Toulon, France.
28. Koncel-Kedziorski, R., Bekal, D., Luan, Y., Lapata, M., and Hajishirzi, H. (2019, January 3–5). Text Generation from Knowledge Graphs with Graph Transformers. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.
29. Xu, X., Liu, C., and Song, D. (2017). Sqlnet: Generating structured queries from natural language without reinforcement learning. arXiv.
30. Koutrika, G., Simitsis, A., and Ioannidis, Y.E. (2010, January 1–6). Explaining structured queries in natural language. Proceedings of the 2010 IEEE 26th International Conference on Data Engineering (ICDE 2010), Long Beach, CA, USA.10.1109/ICDE.2010.5447824 ↗
31. Ngonga Ngomo, A.C., Bühmann, L., Unger, C., Lehmann, J., and Gerber, D. (2013, January 13–17). Sorry, i don’t speak SPARQL: Translating SPARQL queries into natural language. Proceedings of the 22nd International Conference on World Wide Web, Riode Janeiro, Brazil.10.1145/2488388.2488473 ↗
32. Eriguchi, A., Hashimoto, K., and Tsuruoka, Y. (2016). Tree-to-sequence attentional neural machine translation. arXiv.10.18653/v1/P16-1078 ↗
33. Tai, K.S., Socher, R., and Manning, C.D. (2015). Improved semantic representations from tree-structured long short-term memory networks. arXiv.10.3115/v1/P15-1150 ↗
34. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (2017). Graph attention networks. arXiv.
35. Schlichtkrull, M., Kipf, T.N., Bloem, P., Van Den Berg, R., Titov, I., and Welling, M. (2018). Modeling relational data with graph convolutional networks. European Semantic Web Conference, Springer.10.1007/978-3-319-93417-4_38 ↗
36. De Cao, N., Aziz, W., and Titov, I. (2018). Question answering by reasoning across documents with graph convolutional networks. arXiv.10.18653/v1/N19-1240 ↗
37. Shaw, P., Uszkoreit, J., and Vaswani, A. (2018). Self-attention with relative position representations. arXiv.10.18653/v1/N18-2074 ↗
38. Xiao, F., Li, J., Zhao, H., Wang, R., and Chen, K. (2019). Lattice-based transformer encoder for neural machine translation. arXiv.10.18653/v1/P19-1298 ↗
39. Wang, B., Shin, R., Liu, X., Polozov, O., and Richardson, M. (2019). Rat-sql: Relation-aware schema encoding and linking for text-to-sql parsers. arXiv.10.18653/v1/2020.acl-main.677 ↗
40. Li, X., Yan, H., Qiu, X., and Huang, X. (2020). FLAT: Chinese NER Using Flat-Lattice Transformer. arXiv.10.18653/v1/2020.acl-main.611 ↗
41. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, MIT Press.
42. Ba, J.L., Kiros, J.R., and Hinton, G.E. (2016). Layer normalization. arXiv.
43. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
44. See, A., Liu, P.J., and Manning, C.D. (2017). Get to the point: Summarization with pointer-generator networks. arXiv.10.18653/v1/P17-1099 ↗
45. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, Vancouver Convention Center.
46. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
47. Pennington, J., Socher, R., and Manning, C. (2014, January 26–28). GloVe: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1162 ↗
48. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 29–31). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MU, USA.
49. Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 8–10). BLEU: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.10.3115/1073083.1073135 ↗
50. Wołk, K., and Koržinek, D. (2016). Comparison and adaptation of automatic evaluation metrics for quality assessment of re-speaking. arXiv.10.7494/csci.2017.18.2.129 ↗
51. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.