Essential Features in a Theory of Context for Enabling Artificial General Intelligence
Source abstract
Abstract
Despite recent Artificial Intelligence (AI) advances in narrow task areas such as face recognition and natural language processing, the emergence of general machine intelligence continues to be elusive. Such an AI must overcome several challenges, one of which is the ability to be aware of, and appropriately handle, context. In this article, we argue that context needs to be rigorously treated as a first-class citizen in AI research and discourse for achieving true general machine intelligence. Unfortunately, context is only loosely defined, if at all, within AI research. This article aims to synthesize the myriad pragmatic ways in which context has been used, or implicitly assumed, as a core concept in multiple AI sub-areas, such as representation learning and commonsense reasoning. While not all definitions are equivalent, we systematically identify a set of seven features associated with context in these sub-areas. We argue that such features are necessary for a sufficiently rich theory of context, as applicable to practical domains and applications in AI.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Advanced Graph Neural Networks · Data Quality and Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Ste. 1001, Marina del Rey, CA 90292, USA
References (100+)
Show source-provided references
1. Deep learning10.1038/nature14539 ↗
2. Wang, Y.E., Wei, G.Y., and Brooks, D. (2019). Benchmarking tpu, gpu, and cpu platforms for deep learning. arXiv.
3. Rimal, B.P., Choi, E., and Lumb, I. (2009, January 25–27). A taxonomy and survey of cloud computing systems. Proceedings of the 2009 Fifth International Joint Conference on INC, IMS and IDC, Seoul, Korea.10.1109/NCM.2009.218 ↗
4. Benefits, adoption barriers and myths of open data and open government10.1080/10580530.2012.716740 ↗
5. A survey on deep learning based face recognition10.1016/j.cviu.2019.102805 ↗
6. Singh, S.P., Kumar, A., Darbari, H., Singh, L., Rastogi, A., and Jain, S. (2017, January 1–2). Machine translation using deep learning: An overview. Proceedings of the 2017 International Conference on Computer, Communications and Electronics (Comptelix), Jaipur, India.10.1109/COMPTELIX.2017.8003957 ↗
7. Deep learning approaches for question answering system10.1016/j.procs.2018.05.090 ↗
8. Ford, M. (2018). Architects of Intelligence: The Truth about AI from the People Building It, Packt Publishing Ltd.
9. Müller, V.C., and Bostrom, N. (2016). Future progress in artificial intelligence: A survey of expert opinion. Fundamental Issues of Artificial Intelligence, Springer.10.1007/978-3-319-26485-1_33 ↗
10. The Dartmouth College artificial intelligence conference: The next fifty years
11. Mims, C. (2021, September 02). Why Artificial Intelligence Isn’t Intelligent. Available online: https://www.wsj.com/articles/why-artificial-intelligence-isnt-intelligent-11627704050.
12. Parkhi, O.M., Vedaldi, A., and Zisserman, A. (2015, January 7–10). Deep face recognition. Proceedings of the BMVC 2015, Swansea, UK.10.5244/C.29.41 ↗
13. Dehghani, M., Gouws, S., Vinyals, O., Uszkoreit, J., and Kaiser, Ł (2018). Universal transformers. arXiv.
14. Wang, Y., Sun, A., Han, J., Liu, Y., and Zhu, X. (2018, January 23–27). Sentiment analysis by capsules. Proceedings of the 2018 World Wide Web Conference, Lyon, France.10.1145/3178876.3186015 ↗
15. Lockard, C., Shiralkar, P., and Dong, X.L. (2019, January 3–5). Openceres: When open information extraction meets the semi-structured web. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.
16. Irvine, J., Schieffelin, B., Series, C., Goodwin, M.H., Kuipers, J., Kulick, D., Lucy, J., and Ochs, E. (1992). Rethinking Context: Language as an Interactive Phenomenon, Cambridge University Press. Number 11.
17. Context in Artificial Intelligence: I. A survey of the literature
18. A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications10.1207/S15327051HCI16234_02 ↗
19. Integrating Context into Artificial Intelligence: Research from the Robotics Collaborative Technology Alliance
20. Singhal, A., Luo, J., and Zhu, W. (2003, January 18–20). Probabilistic spatial context models for scene content understanding. Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI, USA.
21. Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., and Belongie, S. (2007, January 14–21). Objects in context. Proceedings of the 2007 IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Brazil.10.1109/ICCV.2007.4408986 ↗
22. Zettlemoyer, L.S., and Collins, M. (2009, January 2–7). Learning Context-Dependent Mappings from Sentences to Logical Form. Proceedings of the 47th Annual Meeting of the ACL, Suntec, Singapore.10.3115/1690219.1690283 ↗
23. Heitz, G., and Koller, D. (2008). Learning spatial context: Using stuff to find things. European Conference on Computer Vision, Springer.10.1007/978-3-540-88682-2_4 ↗
24. Grice, H.P. (1975). Logic and conversation. Speech Acts, Brill.10.1163/9789004368811_003 ↗
25. Never-ending learning10.1145/3191513 ↗
26. Chatbots: Are they really useful?
27. Chatbots: Changing user needs and motivations10.1145/3236669 ↗
28. Cognition as context (whose cognition?)10.1207/s15327973rlsi3101_2 ↗
29. Adaptive planning10.1207/s15516709cog1203_3 ↗
30. Has evolution primed humans to “beware the beast”?10.1073/pnas.0707885104 ↗
31. Nature, nurture, and the disunity of knowledge10.1111/j.1749-6632.2001.tb03470.x ↗
32. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
33. Sarker, M.K., Zhou, L., Eberhart, A., and Hitzler, P. (2021). Neuro-Symbolic Artificial Intelligence Current Trends. arXiv.10.3233/AIC-210084 ↗
34. GPT-3: Its nature, scope, limits, and consequences10.1007/s11023-020-09548-1 ↗
35. Coulmas, F., Rabin, C., Ibrahim, M.H., Massamba, D.P.B., Daswani, C.J., Pasierbsky, F., Takada, M., Sugito, S., Porksen, U., and Ehlich, K. (1989). Language Adaptation, Cambridge University Press.
36. Attardo, S. (2017). The Routledge Handbook of Language and Humor, Taylor & Francis.10.4324/9781315731162 ↗
37. The neuroanatomical basis of understanding sarcasm and its relationship to social cognition10.1037/0894-4105.19.3.288 ↗
38. Tagg, C. (2012). Discourse of Text Messaging: Analysis of SMS Communication, Bloomsbury Publishing.
39. Representation learning: A review and new perspectives10.1109/TPAMI.2013.50 ↗
40. An overview on data representation learning: From traditional feature learning to recent deep learning10.1016/j.jfds.2017.05.001 ↗
41. Zheng, A., and Casari, A. (2018). Feature Engineering for Machine Learning: Principles and Techniques for Data Scientists, O’Reilly Media, Inc.
42. Techniques and applications for sentiment analysis10.1145/2436256.2436274 ↗
43. Sharma, A., and Dey, S. (2012, January 23–26). A comparative study of feature selection and machine learning techniques for sentiment analysis. Proceedings of the 2012 ACM Research in Applied Computation Symposium, San Antonio, TX, USA.10.1145/2401603.2401605 ↗
44. Natural language processing (almost) from scratch
45. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013, January 5–10). Distributed representations of words and phrases and their compositionality. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
46. Latent dirichlet allocation
47. Cnn-based joint clustering and representation learning with feature drift compensation for large-scale image data10.1109/TMM.2017.2745702 ↗
48. Joulin, A., Grave, E., Bojanowski, P., and Mikolov, T. (2016). Bag of Tricks for Efficient Text Classification. arXiv.10.18653/v1/E17-2068 ↗
49. Pennington, J., Socher, R., and Manning, C.D. (2014, January 25–29). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1162 ↗
50. Perozzi, B., Al-Rfou, R., and Skiena, S. (2014, January 24–27). Deepwalk: Online learning of social representations. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.10.1145/2623330.2623732 ↗
51. Grover, A., and Leskovec, J. (2016, January 13–17). node2vec: Scalable feature learning for networks. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939754 ↗
52. Kejriwal, M., and Szekely, P. (2017). Neural embeddings for populated geonames locations. International Semantic Web Conference, Springer.10.1007/978-3-319-68204-4_14 ↗
53. Khan, S., Naseer, M., Hayat, M., Zamir, S.W., Khan, F.S., and Shah, M. (2021). Transformers in vision: A survey. arXiv.10.1145/3505244 ↗
54. Attention flows: Analyzing and comparing attention mechanisms in language models10.1109/TVCG.2020.3028976 ↗
55. Minsky, M.L. (2006). The Emotion Machine: Commensense Thinking, Artificial Intelligence, and the Future of the Human Mind, Simon & Schuster. Google-Books-ID: AT0UlQEACAAJ.
56. Commonsense reasoning and commonsense knowledge in artificial intelligence10.1145/2701413 ↗
57. Kejriwal, M. (2021, September 03). An AI Expert Explains Why It’s Hard to Give Computers Something You Take for Granted: Common Sense. Available online: https://theconversation.com/an-ai-expert-explains-why-its-hard-to-give-computers-something-you-take-for-granted-common-sense-165600.
58. Lin, H., and Ng, V. (2019, January 1). Abstractive summarization: A survey of the state of the art. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.10.1609/aaai.v33i01.33019815 ↗
59. Lin, B.Y., Zhou, W., Shen, M., Zhou, P., Bhagavatula, C., Choi, Y., and Ren, X. (2020, January 16–20). CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2020, Online.10.18653/v1/2020.findings-emnlp.165 ↗
60. Gordon, A.S., and Hobbs, J.R. (2017). A Formal Theory of Commonsense Psychology: How People Think People Think, Cambridge University Press.10.1017/9781316584705 ↗
61. Santos, H., Kejriwal, M., Mulvehill, A.M., Forbush, G., McGuinness, D.L., and Rivera, A.R. (2021). An experimental study measuring human annotator categorization agreement on commonsense sentences. Exp. Results, 2.10.1017/exp.2021.9 ↗
62. Koehler, J., Nebel, B., Hoffmann, J., and Dimopoulos, Y. (1997). Extending planning graphs to an ADL subset. European Conference on Planning, Springer.10.1007/3-540-63912-8_92 ↗
63. Koenig, S., and Simmons, R.G. (1994). Risk-sensitive planning with probabilistic decision graphs. Principles of Knowledge Representation and Reasoning, Elsevier.10.1016/B978-1-4832-1452-8.50129-9 ↗
64. Introducing the knowledge graph: Things, not strings
65. Bellomarini, L., Fakhoury, D., Gottlob, G., and Sallinger, E. (2019, January 8–12). Knowledge graphs and enterprise AI: The promise of an enabling technology. Proceedings of the 2019 IEEE 35th International Conference on Data Engineering (ICDE), Macau, China.10.1109/ICDE.2019.00011 ↗
66. Xu, D., Ruan, C., Korpeoglu, E., Kumar, S., and Achan, K. (2020, January 3–7). Product knowledge graph embedding for e-commerce. Proceedings of the 13th International Conference on Web Search and Data Mining, Houston, TX, USA.10.1145/3336191.3371778 ↗
67. Knowledge Graphs and COVID-19: Opportunities, Challenges, and Implementation
68. Kejriwal, M. (2019). Domain-Specific Knowledge Graph Construction, Springer.10.1007/978-3-030-12375-8 ↗
69. Kannan, A.V., Fradkin, D., Akrotirianakis, I., Kulahcioglu, T., Canedo, A., Roy, A., Yu, S.Y., Arnav, M., and Al Faruque, M.A. (2020, January 19–23). Multimodal knowledge graph for deep learning papers and code. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Online.10.1145/3340531.3417439 ↗
70. OWL web ontology language overview
71. The semantic web10.1038/scientificamerican0501-34 ↗
72. Ilievski, F., Szekely, P., and Kejriwal, M. (2021, September 02). Commonsense Knowledge Graphs (CSKGs). Available online: https://usc-isi-i2.github.io/ISWC20/.
73. Garcez, A.D., Gori, M., Lamb, L.C., Serafini, L., Spranger, M., and Tran, S.N. (2019). Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. arXiv.
74. Toward human-understandable, explainable AI10.1109/MC.2018.3620965 ↗
75. Hoffman, R.R., Mueller, S.T., Klein, G., and Litman, J. (2018). Metrics for explainable AI: Challenges and prospects. arXiv.
76. Turek, M. (2021, September 02). Explainable Artificial Intelligence (XAI). Available online: https://www.darpa.mil/program/explainable-artificial-intelligence.
77. Došilović, F.K., Brčić, M., and Hlupić, N. (2018, January 21–25). Explainable artificial intelligence: A survey. Proceedings of the 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia.10.23919/MIPRO.2018.8400040 ↗
78. Valentino, M., Thayaparan, M., and Freitas, A. (2020). Explainable natural language reasoning via conceptual unification. arXiv.
79. Motif-based embedding for graph clustering10.1088/1742-5468/2016/12/123401 ↗
80. Zhang, S., Hu, Z., Subramonian, A., and Sun, Y. (2020). Motif-driven contrastive learning of graph representations. arXiv.
81. Garnham, A. (1997). Representing information in mental models. Cognitive Models of Memory, The MIT Press. Studies in Cognition.
82. Dhingra, B., Zhou, Z., Fitzpatrick, D., Muehl, M., and Cohen, W.W. (2016). Tweet2vec: Character-Based Distributed Representations for Social Media. arXiv.10.18653/v1/P16-2044 ↗
83. Wang, L., Gao, C., Wei, J., Ma, W., Liu, R., and Vosoughi, S. (2020). An Empirical Survey of Unsupervised Text Representation Methods on Twitter Data. arXiv.10.18653/v1/2020.wnut-1.27 ↗
84. Sehwag, V., Bhagoji, A.N., Song, L., Sitawarin, C., Cullina, D., Chiang, M., and Mittal, P. (2019, January 15). Analyzing the robustness of open-world machine learning. Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security, London, UK.10.1145/3338501.3357372 ↗
85. Cambel, A.B. (1993). Applied Chaos Theory: A Paradigm for Complexity, Elsevier.
86. Living with uncertainty: The moral significance of ignorance10.1093/analys/anp111 ↗
87. Keefe, R., and Smith, P. (1997). Vagueness by degrees. Vagueness: A Reader, MIT Press.
88. Circularity and induction10.1093/analys/23.6.123 ↗
89. Senator, T. (2021, September 03). Science of Artificial Intelligence and Learning for Open-World Novelty (SAIL-ON). Available online: https://www.darpa.mil/program/science-of-artificial-intelligence-and-learning-for-open-world-novelty.
90. Some applications of minimal open sets10.1155/S0161171201006482 ↗
91. Average cost optimal stationary policies in infinite state Markov decision processes with unbounded costs10.1287/opre.37.4.626 ↗
92. Langley, P. (2020, January 7–12). Open-world learning for radically autonomous agents. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.
93. Kotz, A.M., Dittrich, K.R., and Mulle, J.A. (1988). Supporting semantic rules by a generalized event/trigger mechanism. International Conference on Extending Database Technology, Springer.10.1007/3-540-19074-0_48 ↗
94. Batista-Navarro, R.T., Kontonatsios, G., Mihăilă, C., Thompson, P., Rak, R., Nawaz, R., Korkontzelos, I., and Ananiadou, S. (2013). Facilitating the analysis of discourse phenomena in an interoperable NLP platform. International Conference on Intelligent Text Processing and Computational Linguistics, Springer.10.1007/978-3-642-37247-6_45 ↗
95. Keren, G. (2011). Perspectives on Framing, Psychology Press.10.4324/9780203854167 ↗
96. Kühberger, A. (2017). Framing. Cognitive Illusions: Intriguing Phenomena in Judgement, Thinking and Memory, Psychology Press. [2nd ed.].
97. Configurable workflow models10.1142/S0218843008001798 ↗
98. Van Der Aalst, W., Van Hee, K.M., and van Hee, K. (2004). Workflow Management: Models, Methods, and Systems, MIT Press.
99. NSF (2021, September 02). National Artificial Intelligence (AI) Research Institutes, Available online: https://beta.nsf.gov/funding/opportunities/national-artificial-intelligence-research-institutes.
100. AGRA: AI-augmented geographic routing approach for IoT-based incident-supporting applications10.1016/j.future.2017.08.009 ↗