Quantum Natural Language Processing: Challenges and Opportunities
Source abstract
Abstract
The meeting between Natural Language Processing (NLP) and Quantum Computing has been very successful in recent years, leading to the development of several approaches of the so-called Quantum Natural Language Processing (QNLP). This is a hybrid field in which the potential of quantum mechanics is exploited and applied to critical aspects of language processing, involving different NLP tasks. Approaches developed so far span from those that demonstrate the quantum advantage only at the theoretical level to the ones implementing algorithms on quantum hardware. This paper aims to list the approaches developed so far, categorizing them by type, i.e., theoretical work and those implemented on classical or quantum hardware; by task, i.e., general purpose such as syntax-semantic representation or specific NLP tasks, like sentiment analysis or question answering; and by the resource used in the evaluation phase, i.e., whether a benchmark dataset or a custom one has been used. The advantages offered by QNLP are discussed, both in terms of performance and methodology, and some considerations about the possible usage QNLP approaches in the place of state-of-the-art deep learning-based ones are given.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Materials Science · Topic Modeling · Quantum Computing Algorithms and Architecture
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute for High Performance Computing and Networking (ICAR), National Research Council (CNR), 80131 Naples, Italy
References (100+)
Show source-provided references
1. Attention is all you need
2. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2–7). Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.
3. Language models are unsupervised multitask learners
4. Clark, K., Luong, M.T., Le, Q.V., and Manning, C.D. (2020, January 26–30). ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. Proceedings of the ICLR, Addis Ababa, Ethiopia.
5. Zhu, J., Xia, Y., Wu, L., He, D., Qin, T., Zhou, W., Li, H., and Liu, T. (2019, January 6–9). Incorporating BERT into Neural Machine Translation. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA.
6. Sun, C., Qiu, X., Xu, Y., and Huang, X. (2019, January 18–20). How to fine-tune bert for text classification?. Proceedings of the China National Conference on Chinese Computational Linguistics, Kunming, China.10.1007/978-3-030-32381-3_16 ↗
7. Lee, K., He, L., Lewis, M., and Zettlemoyer, L. (2017, January 9–11). End-to-end Neural Coreference Resolution. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.10.18653/v1/D17-1018 ↗
8. ELECTRA for Neural Coreference Resolution in Italian10.1109/ACCESS.2021.3105278 ↗
9. BERT syntactic transfer: A computational experiment on Italian, French and English languages10.1016/j.csl.2021.101261 ↗
10. Assessing BERT’s ability to learn Italian syntax: A study on null-subject and agreement phenomena
11. Chi, E.A., Hewitt, J., and Manning, C.D. (2020). Finding universal grammatical relations in multilingual BERT. arXiv.10.18653/v1/2020.acl-main.493 ↗
12. Language Models are Few-Shot Learners
13. GPT-3: Its nature, scope, limits, and consequences10.1007/s11023-020-09548-1 ↗
14. Jawahar, G., Sagot, B., and Seddah, D. (August, January 28). What does BERT learn about the structure of language?. Proceedings of the ACL 2019-57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.10.18653/v1/P19-1356 ↗
15. Liu, X., Zheng, Y., Du, Z., Ding, M., Qian, Y., Yang, Z., and Tang, J. (2021). GPT understands, too. arXiv.
16. A primer in bertology: What we know about how bert works10.1162/tacl_a_00349 ↗
17. How can we know what language models know?10.1162/tacl_a_00324 ↗
18. Quantum computation and quantum information10.1119/1.1463744 ↗
19. The Effect of Pauli gates on the superposition for four-qubit in Bloch sphere
20. Ben-David, S., Childs, A.M., Gilyén, A., Kretschmer, W., Podder, S., and Wang, D. (2020, January 16–19). Symmetries, graph properties, and quantum speedups. Proceedings of the 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS), Durham, NC, USA.10.1109/FOCS46700.2020.00066 ↗
21. Quantum machine learning10.1038/nature23474 ↗
22. Machine learning: Quantum vs classical10.1109/ACCESS.2020.3041719 ↗
23. A non-review of Quantum Machine Learning: Trends and explorations10.22331/qv-2020-03-17-32 ↗
24. Coecke, B., de Felice, G., Meichanetzidis, K., Toumi, A., Gogioso, S., and Chiappori, N. (2022, June 01). Quantum Natural Language Processing. Available online: http://www.cs.ox.ac.uk/people/bob.coecke/QNLP-ACT.pdf.
25. Coecke, B., Sadrzadeh, M., and Clark, S. (2010). Mathematical foundations for a compositional distributional model of meaning. arXiv.
26. Quantum Algorithms for Compositional Natural Language Processing10.4204/EPTCS.221.8 ↗
27. Near-term quantum algorithms for quantum many-body systems10.1088/1742-6596/1290/1/012003 ↗
28. Wu, S., Li, J., Zhang, P., and Zhang, Y. (2021, January 7–11). Natural Language Processing Meets Quantum Physics: A Survey and Categorization. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online.10.18653/v1/2021.emnlp-main.254 ↗
29. Towards a cognitive analysis of polysemy, ambiguity, and vagueness
30. Gries, S.T. (2019). Polysemy. Cognitive Linguistics—Key Topics, De Gruyter Mouton.10.1515/9783110626438-002 ↗
31. Coecke, B., de Felice, G., Meichanetzidis, K., and Toumi, A. (2020). Foundations for Near-Term Quantum Natural Language Processing. arXiv.
32. von Neumann, J. (1955). Mathematical foundations of quantum mechanics. Investigations in Physics, Princeton University Press.
33. Quantum Computing in the NISQ era and beyond10.22331/q-2018-08-06-79 ↗
34. Quantum random access memory10.1103/PhysRevLett.100.160501 ↗
35. Grover, L.K. (1996, January 22–24). A fast quantum mechanical algorithm for database search. Proceedings of the Twenty-Eighth Annual ACM Symposium on Theory of Computing, Philadelphia, PA, USA.10.1145/237814.237866 ↗
36. Kindergarten quantum mechanics: Lecture notes10.1063/1.2158713 ↗
37. Coecke, B., and Kissinger, A. (2018, January 18–22). Picturing quantum processes. Proceedings of the International Conference on Theory and Application of Diagrams, Edinburgh, UK.10.1007/978-3-319-91376-6_6 ↗
38. Distributional structure10.1080/00437956.1954.11659520 ↗
39. Steedman, M., and Baldridge, J. (2011). Combinatory categorial grammar. Non-Transformational Syntax: Formal and Explicit Models of Grammar, Wiley-Blackwell.10.1002/9781444395037.ch5 ↗
40. Analysis of explainers of black box deep neural networks for computer vision: A survey10.3390/make3040048 ↗
41. Universal grammar
42. Die syntaktische Konnexit ät
43. A quasi-arithmetical notation for syntactic description10.2307/410452 ↗
44. The mathematics of sentence structure10.1080/00029890.1958.11989160 ↗
45. Lambek, J. (1997, January 22–24). Type grammar revisited. Proceedings of the International Conference on Logical Aspects of Computational Linguistics, Nancy, France.
46. Clark, S., Coecke, B., and Sadrzadeh, M. (2008, January 26–28). A compositional distributional model of meaning. Proceedings of the Second Quantum Interaction Symposium (QI-2008), Oxford, UK.
47. Buszkowski, W. (2001, January 27–29). Lambek grammars based on pregroups. Proceedings of the International Conference on Logical Aspects of Computational Linguistics, Le Croisic, France.10.1007/3-540-48199-0_6 ↗
48. Categorical quantum mechanics
49. Quantum Natural Language Processing on Near-Term Quantum Computers10.4204/EPTCS.340.11 ↗
50. Pestun, V., and Vlassopoulos, Y. (2017). Tensor network language model. arXiv.
51. Correia, A., Moortgat, M., and Stoof, H. (2021). Grover’s Algorithm for Question Answering. arXiv.
52. Application of quantum natural language processing for language translation10.1109/ACCESS.2021.3108768 ↗
53. Sordoni, A., Nie, J.Y., and Bengio, Y. (August, January 28). Modeling term dependencies with quantum language models for ir. Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, Dublin, Ireland.
54. Xie, M., Hou, Y., Zhang, P., Li, J., Li, W., and Song, D. (2015, January 25–31). Modeling quantum entanglements in quantum language models. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina.
55. Li, Q., Melucci, M., and Tiwari, P. (2018, January 14–17). Quantum language model-based query expansion. Proceedings of the 2018 ACM SIGIR International Conference on Theory of Information Retrieval, Tianjin, China.10.1145/3234944.3234970 ↗
56. Jiang, Y., Zhang, P., Gao, H., and Song, D. (2020, January 25–30). A quantum interference inspired neural matching model for ad-hoc retrieval. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual.10.1145/3397271.3401070 ↗
57. Zhang, P., Niu, J., Su, Z., Wang, B., Ma, L., and Song, D. (2018, January 2–7). End-to-end quantum-like language models with application to question answering. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11979 ↗
58. Zhang, P., Su, Z., Zhang, L., Wang, B., and Song, D. (2018, January 22–26). A quantum many-body wave function inspired language modeling approach. Proceedings of the 27th ACM International Conference on Information and Knowledge Management, Torino, Italy.10.1145/3269206.3271723 ↗
59. Li, Q., Wang, B., and Melucci, M. (2019, January 2–7). CNM: An Interpretable Complex-valued Network for Matching. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.
60. Zhang, Y., Song, D., Li, X., and Zhang, P. (2018, January 26–29). Unsupervised sentiment analysis of twitter posts using density matrix representation. Proceedings of the European Conference on Information Retrieval, Grenoble, France.10.1007/978-3-319-76941-7_24 ↗
61. Zhang, Y., Li, Q., Song, D., Zhang, P., and Wang, P. (2019, January 10–16). Quantum-Inspired Interactive Networks for Conversational Sentiment Analysis. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, Macao, China.10.24963/ijcai.2019/755 ↗
62. Zhang, P., Zhang, J., Ma, X., Rao, S., Tian, G., and Wang, J. (2021, January 4). TextTN: Probabilistic Encoding of Language on Tensor Network. Proceedings of the International Conference on Learning Representations, Vienna, Austria.
63. Meichanetzidis, K., Toumi, A., de Felice, G., and Coecke, B. (2020). Grammar-Aware Question-Answering on Quantum Computers. arXiv.
64. Lorenz, R., Pearson, A., Meichanetzidis, K., Kartsaklis, D., and Coecke, B. (2021). Qnlp in practice: Running compositional models of meaning on a quantum computer. arXiv.
65. Vicente Nieto, I. (2021). Towards Machine Translation with Quantum Computers. [Master’s Thesis, University of Stockholm].
66. Mitchell, J., and Lapata, M. (2008, January 15–20). Vector-based models of semantic composition. Proceedings of the ACL-08: HLT, Columbus, OH, USA.
67. Grefenstette, E., and Sadrzadeh, M. (2011, January 27–31). Experimental Support for a Categorical Compositional Distributional Model of Meaning. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, Emnlp 2011, Scotland, UK.
68. BNC Consortium (2007). British national corpus. Oxford Text Archive Core Collection, University of Oxford.
69. Lewis, M. (2020). Towards logical negation for compositional distributional semantics. IfCoLoG J. Logics Their Appl., 7.
70. Applying Distributional Compositional Categorical Models of Meaning to Language Translation10.4204/EPTCS.283.3 ↗
71. The Frobenius anatomy of word meanings I: Subject and object relative pronouns10.1093/logcom/ext044 ↗
72. A new description of orthogonal bases10.1017/S0960129512000047 ↗
73. Kartsaklis, D., and Sadrzadeh, M. (2013, January 18–21). Prior disambiguation of word tensors for constructing sentence vectors. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA.
74. Automatic word sense discrimination
75. A quantum teleportation inspired algorithm produces sentence meaning from word meaning and grammatical structure
76. Interacting quantum observables: Categorical algebra and diagrammatics10.1088/1367-2630/13/4/043016 ↗
77. Parameterized quantum circuits as machine learning models10.1088/2058-9565/ab4eb5 ↗
78. Mining pure high-order word associations via information geometry for information retrieval10.1145/2493175.2493177 ↗
79. Metzler, D., and Croft, W.B. (2005, January 15–19). A markov random field model for term dependencies. Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Salvador, Brazil.10.1145/1076034.1076115 ↗
80. Zhang, P., Li, J., Wang, B., Zhao, X., Song, D., Hou, Y., and Melucci, M. (2016). A quantum query expansion approach for session search. Entropy, 18.10.3390/e18040146 ↗
81. A quasi-current representation for information needs inspired by Two-State Vector Formalism10.1016/j.physa.2017.04.145 ↗
82. Robertson, S.E., and Walker, S. (1994, January 1). Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. Proceedings of the SIGIR’94, Dublin, Ireland.10.1007/978-1-4471-2099-5_24 ↗
83. Van Rijsbergen, C.J. (2004). The Geometry of Information Retrieval, Cambridge University Press.10.1017/CBO9780511543333 ↗
84. Wang, M., Smith, N.A., and Mitamura, T. (2007, January 28–30). What is the Jeopardy model? A quasi-synchronous grammar for QA. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), Prague, Czech Republic.
85. Yang, Y., Yih, W.t., and Meek, C. (2015, January 17–21). Wikiqa: A challenge dataset for open-domain question answering. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.10.18653/v1/D15-1237 ↗
86. Yang, L., Ai, Q., Guo, J., and Croft, W.B. (2016, January 24–28). anmm: Ranking short answer texts with attention-based neural matching model. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, Indianapolis, IN, USA.10.1145/2983323.2983818 ↗
87. Abcnn: Attention-based convolutional neural network for modeling sentence pairs10.1162/tacl_a_00097 ↗
88. Severyn, A., and Moschitti, A. (2015, January 9–13). Learning to rank short text pairs with convolutional deep neural networks. Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile.10.1145/2766462.2767738 ↗
89. Kim, Y. (2014, January 25–29). Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1181 ↗
90. Poria, S., Cambria, E., Hazarika, D., Majumder, N., Zadeh, A., and Morency, L.P. (August, January 30). Context-dependent sentiment analysis in user-generated videos. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, BC, Canada.
91. Glasser, I., Sweke, R., Pancotti, N., Eisert, J., and Cirac, I. (2019). Expressive power of tensor-network factorizations for probabilistic modeling. Adv. Neural Inf. Process. Syst., 32.
92. Dai, B., Li, J., and Xu, R. (2020, January 7–12). Multiple positional self-attention network for text classification. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.
93. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C.D., Ng, A., and Potts, C. (2013, January 18–21). Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA.
94. Pang, B., and Lee, L. (2004, January 21–26). A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts. Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), Barcelona, Spain.10.3115/1218955.1218990 ↗
95. Di Sipio, R., Huang, J.H., Chen, S.Y.C., Mangini, S., and Worring, M. (2021). The Dawn of Quantum Natural Language Processing. arXiv.10.1109/ICASSP43922.2022.9747675 ↗
96. Basile, I., and Tamburini, F. (2017, January 7–11). Towards quantum language models. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.10.18653/v1/D17-1196 ↗
97. Baez, J., and Stay, M. (2010). Physics, topology, logic and computation: A Rosetta Stone. New Structures for Physics, Springer.10.1007/978-3-642-12821-9_2 ↗
98. Selinger, P. (2010). A survey of graphical languages for monoidal categories. New Structures for Physics, Springer.10.1007/978-3-642-12821-9_4 ↗
99. Three models for the description of language10.1109/TIT.1956.1056813 ↗
100. A one-measurement form of simultaneous perturbation stochastic approximation10.1016/S0005-1098(96)00149-5 ↗