Ontology-Based Knowledge Representation in Robotic Systems: A Survey Oriented toward Applications
Source abstract
Abstract
Knowledge representation in autonomous robots with social roles has steadily gained importance through their supportive task assistance in domestic, hospital, and industrial activities. For active assistance, these robots must process semantic knowledge to perform the task more efficiently. In this context, ontology-based knowledge representation and reasoning (KR & R) techniques appear as a powerful tool and provide sophisticated domain knowledge for processing complex robotic tasks in a real-world environment. In this article, we surveyed ontology-based semantic representation unified into the current state of robotic knowledge base systems, with our aim being three-fold: (i) to present the recent developments in ontology-based knowledge representation systems that have led to the effective solutions of real-world robotic applications; (ii) to review the selected knowledge-based systems in seven dimensions: application, idea, development tools, architecture, ontology scope, reasoning scope, and limitations; (iii) to pin-down lessons learned from the review of existing knowledge-based systems for designing better solutions and delineating research limitations that might be addressed in future studies. This survey article concludes with a discussion of future research challenges that can serve as a guide to those who are interested in working on the ontology-based semantic knowledge representation systems for autonomous robots.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Semantic Web and Ontologies · AI-based Problem Solving and Planning · Logic, Reasoning, and Knowledge
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical and Computer Engineering, College of Information and Communication Engineering, Sungkyunkwan University, Suwon 16419, Korea
Funding
Ministry of Trade, Industry and Energy · This research was supported by Korea Evaluation Institute of Industrial Technology(KEIT) fundedby the Ministry of Trade, Industry & Energy (MOTIE) (No.) · Korea Evaluation Institute of Industrial Technology
References (100+)
Show source-provided references
1. Olszewska, J.I., Barreto, M., Bermejo-Alonso, J., Carbonera, J., Chibani, A., Fiorini, S., Goncalves, P., Habib, M., Khamis, A., and Olivares, A. (2017, January 28). Ontology for autonomous robotics. Proceedings of the 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Lisbon, Portugal.10.1109/ROMAN.2017.8172300 ↗
2. Requirements for building an ontology for autonomous robots10.1108/IR-02-2016-0059 ↗
3. Value of social robots in services: Social cognition perspective10.1108/JSM-02-2018-0080 ↗
4. Ontology-based unified robot knowledge for service robots in indoor environments10.1109/TSMCA.2010.2076404 ↗
5. The use of ontologies for effective knowledge modelling and information retrieval10.1016/j.aci.2017.07.003 ↗
6. A review and comparison of ontology-based approaches to robot autonomy10.1017/S0269888919000237 ↗
7. Topp, E.A., Stenmark, M., Ganslandt, A., Svensson, A., Haage, M., and Malec, J. (2018, January 1–5). Ontology-Based Knowledge Representation for Increased Skill Reusability in Industrial Robots. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8593566 ↗
8. Azevedo, H., Belo, J.P.R., and Romero, R.A. (2018, January 6–10). OntPercept: A Perception Ontology for Robotic Systems. Proceedings of the 2018 IEEE Latin American Robotic Symposium, 2018 Brazilian Symposium on Robotics (SBR) and 2018 Workshop on Robotics in Education (WRE), Joao Pessoa, Brazil.10.1109/LARS/SBR/WRE.2018.00089 ↗
9. Joo, S.H., Manzoor, S., Rocha, Y.G., Bae, S.H., Lee, K.H., Kuc, T.Y., and Kim, M. (2020). Autonomous navigation framework for intelligent robots based on a semantic environment modeling. Appl. Sci., 10.10.3390/app10093219 ↗
10. Manzoor, S., Joo, S.H., Rocha, Y.G., Lee, H.U., and Kuc, T.Y. (2019, January 8). A Novel Semantic SLAM Framework for Humanlike High-Level Interaction and Planning in Global Environment. Proceedings of the 1st International Workshop on the Semantic Descriptor, Semantic Modeling and Mapping for Humanlike Perception and Navigation of Mobile Robots toward Large Scale Long-Term Autonomy (SDMM1), Macau, China.
11. Cognition-enabled robot manipulation in human environments: Requirements, recent work, and open problems10.1109/MRA.2016.2616538 ↗
12. Perzylo, A., Somani, N., Profanter, S., Kessler, I., Rickert, M., and Knoll, A. (2016, January 9–14). Intuitive instruction of industrial robots: Semantic process descriptions for small lot production. Proceedings of the 2016 IEEE/rsj International Conference on Intelligent Robots and Systems (IROS), Daejeon Convention Center, DaeJeon, Korea.10.1109/IROS.2016.7759358 ↗
13. Roboearth10.1109/MRA.2011.941632 ↗
14. KnowRob: A knowledge processing infrastructure for cognition-enabled robots10.1177/0278364913481635 ↗
15. Beetz, M., Tenorth, M., and Winkler, J. (2015, January 25–30). Open-ease. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Washington, DC, USA.10.1109/ICRA.2015.7139458 ↗
16. Saxena, A., Jain, A., Sener, O., Jami, A., Misra, D.K., and Koppula, H.S. (2014). Robobrain: Large-scale knowledge engine for robots. arXiv.
17. Daruna, A., Liu, W., Kira, Z., and Chetnova, S. (2019, January 20–24). Robocse: Robot common sense embedding. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.10.1109/ICRA.2019.8794070 ↗
18. Self-Learning Ontology For Instance Segmentation of 3d Indoor Point Cloud10.5194/isprs-archives-XLIII-B2-2020-309-2020 ↗
19. Kanjaruek, S., Li, D., Qiu, R., and Boonsim, N. (2015, January 6–9). Automated ontology framework for service robots. Proceedings of the 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), Zhuhai, China.10.1109/ROBIO.2015.7418770 ↗
20. Kanjaruek, S., and Li, D. (2017, January 21–23). Tracking Objects Robot for healthcare environments. Proceedings of the 2017 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), Exeter, UK.10.1109/iThings-GreenCom-CPSCom-SmartData.2017.95 ↗
21. How to encode semantic knowledge: A method for meaning representation and computer-aided acquisition
22. Toward a standard ontology of surgical process models10.1007/s11548-018-1824-5 ↗
23. Björkelund, A., Bruyninckx, H., Malec, J., Nilsson, K., and Nugues, P. (2012, January 26). Knowledge for Intelligent Industrial Robots. Proceedings of the AAAI Spring Symposium on Designing Intelligent Robots: Reintegrating AI, Palo Alto, CA, USA.
24. Industry 5.0 and human-robot co-working10.1016/j.procs.2019.09.104 ↗
25. On the way from Industry 4.0 to Industry 5.0: From digital manufacturing to digital society
26. Sun, X., and Zhang, Y. (2019, January 12–15). A Review of Domain Knowledge Representation for Robot Task Planning. Proceedings of the 2019 4th International Conference on Mathematics and Artificial Intelligence, Chengdu, China.10.1145/3325730.3325756 ↗
27. Thosar, M., Zug, S., Skaria, A.M., and Jain, A. (2021, April 29). A Review of Knowledge Bases for Service Robots in Household Environments. Available online: https://www.researchgate.net/publication/328249457_A_Review_of_Knowledge_Bases_for_Service_Robots_in_Household_Environments.
28. Gouidis, F., Vassiliades, A., Patkos, T., Argyros, A., Bassiliades, N., and Plexousakis, D. (2019). A review on intelligent object perception methods combining knowledge-based reasoning and machine learning. arXiv.
29. The role of ontologies in autonomic computing systems10.1147/sj.433.0598 ↗
30. Bermejo-Alonso, J., Sanz, R., Rodríguez, M., and Hernández, C. (2010, January 7–13). Ontology-based engineering of autonomous systems. Proceedings of the 2010 IEEE Sixth International Conference on Autonomic and Autonomous Systems, Cancun, Mexico.10.1109/ICAS.2010.15 ↗
31. Bermejo-Alonso, J., Hernández, C., and Sanz, R. (2016, January 21–26). Model-based engineering of autonomous systems using ontologies and metamodels. Proceedings of the 2016 IEEE International Symposium on Systems Engineering (ISSE), Edinburgh, UK.10.1109/SysEng.2016.7753185 ↗
32. Bermejo-Alonso, J., Sanz, R., Rodríguez, M., and Hernández, C. (2011). Ontology engineering for the autonomous systems domain. International Joint Conference on Knowledge Discovery, Knowledge Engineering, and Knowledge Management, Springer.
33. The exchange of knowledge using cloud robotics10.1109/LRA.2018.2794626 ↗
34. de Freitas, E.P., Olszewska, J.I., Carbonera, J.L., Fiorini, S.R., Khamis, A., Ragavan, S.V., Barreto, M.E., Prestes, E., Habib, M.K., and Redfield, S. (2020). Ontological concepts for information sharing in cloud robotics. J. Ambient. Intell. Humaniz. Comput., 1–12.
35. RehabRobo-Onto: Design, development and maintenance of a rehabilitation robotics ontology on the cloud10.1016/j.rcim.2014.08.010 ↗
36. Balakirsky, S., Kootbally, Z., Schlenoff, C., Kramer, T., and Gupta, S. (2012, January 7–12). An industrial robotic knowledge representation for kit building applications. Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.10.1109/IROS.2012.6385871 ↗
37. Schäfer, F., Kriesten, R., Chrenko, D., and Gechter, F. (2017, January 27–29). No need to learn from each other? Potentials of knowledge modeling in autonomous vehicle systems engineering towards new methods in multidisciplinary contexts. Proceedings of the 2017 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC), Madeira, Portugal.10.1109/ICE.2017.8279921 ↗
38. Jäger, G., Mueller, C.A., Thosar, M., Zug, S., and Birk, A. (2018). Towards robot-centric conceptual knowledge acquisition. arXiv.
39. Beßler, D., Koralewski, S., and Beetz, M. (2021, April 29). Knowledge Representation for Cognition- and Learning-enabled Robot Manipulation. Available online: https://www.semanticscholar.org/paper/Knowledge-Representation-for-Cognition-and-Robot-Be%C3%9Fler-Koralewski/a912517f69db6dd78f80c249320d5a781a67a70d.
40. Fischer, L., Hasler, S., Deigmöller, J., Schnürer, T., Redert, M., Pluntke, U., Nagel, K., Senzel, C., Ploennigs, J., and Richter, A. (2021, April 29). Which Tool to Use? Grounded Reasoning in Everyday Environments with Assistant Robots. CogRob@ KR. Available online: https://www.semanticscholar.org/paper/Which-tool-to-use-Grounded-reasoning-in-everyday-Fischer-Hasler/25c3841a905553a370f89d657f3376f63207dc3b.
41. Pinacho, L.S., Wich, A., Yazdani, F., and Beetz, M. (2018). Acquiring knowledge of object arrangements from human examples for household robots. Joint German/Austrian Conference on Artificial Intelligence (Künstliche Intelligenz), Springer.
42. Desire-driven reasoning for personal care robots10.1109/ACCESS.2019.2921112 ↗
43. Vassiliades, A., Bassiliades, N., Gouidis, F., and Patkos, T. (2020). A Knowledge Retrieval Framework for Household Objects and Actions with External Knowledge. InInternational Conference on Semantic Systems, Springer.10.1007/978-3-030-59833-4_3 ↗
44. Gehrig, D., Krauthausen, P., Rybok, L., Kuehne, H., Hanebeck, U.D., Schultz, T., and Stiefelhagen, R. (2011, January 25–30). Combined intention, activity, and motion recognition for a humanoid household robot. Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA.10.1109/IROS.2011.6048716 ↗
45. Patterson, D.J., Fox, D., Kautz, H., and Philipose, M. (2005, January 18–21). Fine-grained activity recognition by aggregating abstract object usage. Proceedings of the Ninth IEEE International Symposium on Wearable Computers (ISWC’05), Osaka, Japan.
46. Transferring skills to humanoid robots by extracting semantic representations from observations of human activities10.1016/j.artint.2015.08.009 ↗
47. Artificial cognition for social human–robot interaction: An implementation10.1016/j.artint.2016.07.002 ↗
48. Learning weakly correlated cause–effects for gardening with a cognitive system10.1016/j.engappai.2014.07.017 ↗
49. Efficient interactive decision making framework for robotic applications10.1016/j.artint.2015.04.004 ↗
50. Potaov, A. (2021, March 08). Enabling Cognitive Visual Question Answering. Available online: https://blog.singularitynet.io/enabling-cognitive-visual-question-answering-a93febd454a7.
51. He, B., Xia, M., Yu, X., Jian, P., Meng, H., and Chen, Z. (2017, January 29–31). An educational robot system of visual question answering for preschoolers. Proceedings of the 2017 IEEE 2nd International Conference on Robotics and Automation Engineering (ICRAE), Shanghai, China.10.1109/ICRAE.2017.8291426 ↗
52. Kindergarten Social Assistive Robot (KindSAR) for children’s geometric thinking and metacognitive development in preschool education: A pilot study10.1016/j.chb.2014.03.009 ↗
53. Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv.
54. Li, G., Su, H., and Zhu, W. (2017). Incorporating external knowledge to answer open-domain visual questions with dynamic memory networks. arXiv.
55. Narasimhan, M., and Schwing, A.G. (2018, January 8–14). Straight to the facts: Learning knowledge base retrieval for factual visual question answering. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01237-3_28 ↗
56. Wang, P., Wu, Q., Shen, C., Hengel, A.v.d., and Dick, A. (2015). Explicit knowledge-based reasoning for visual question answering. arXiv.
57. Wu, Q., Wang, P., Shen, C., Dick, A., and Van Den Hengel, A. (2016, January 27–30). Ask me anything: Free-form visual question answering based on knowledge from external sources. Proceedings of the IEEE Conference on Computer Vision and Pattern, Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.500 ↗
58. Shah, S., Mishra, A., Yadati, N., and Talukdar, P.P. (February, January 27). Kvqa: Knowledge-aware visual question answering. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, Hi, USA.
59. Visual question answering: A survey of methods and datasets10.1016/j.cviu.2017.05.001 ↗
60. Malinowski, M., Rohrbach, M., and Fritz, M. (2015, January 7–13). Ask your neurons: A neural-based approach to answering questions about images. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.9 ↗
61. Ma, L., Lu, Z., and Li, H. (2016, January 12–17). Learning to answer questions from image using convolutional neural network. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10442 ↗
62. Zhu, Y., Groth, O., Bernstein, M., and Fei-Fei, L. (2016, January 27–30). Visual7w: Grounded question answering in images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.540 ↗
63. Xu, H., and Saenko, K. (2016). Ask, attend and answer: Exploring question-guided spatial attention for visual question answering. InEuropean Conference on Computer Vision, Springer.10.1007/978-3-319-46478-7_28 ↗
64. Chen, K., Wang, J., Chen, L.C., Gao, H., Xu, W., and Nevatia, R. (2015). Abc-cnn: An attention based convolutional neural network for visual question answering. arXiv.
65. Yang, Z., He, X., Gao, J., Deng, L., and Smola, A. (2016, January 27–30). Stacked attention networks for image question answering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.10 ↗
66. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
67. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014). Microsoft coco: Common objects in context. InEuropean Conference on Computer Vision, Springer.10.1007/978-3-319-10602-1_48 ↗
68. Beetz, M., Beßler, D., Haidu, A., Pomarlan, M., Bozcuoğlu, A.K., and Bartels, G. (2018, January 21–25). Know rob 2.0—A 2nd generation knowledge processing framework for cognition-enabled robotic agents. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.10.1109/ICRA.2018.8460964 ↗
69. Knowledge representation applied to robotic orthopedic surgery10.1016/j.rcim.2014.08.014 ↗
70. Bruno, B., Chong, N.Y., Kamide, H., Kanoria, S., Lee, J., Lim, Y., Pandey, A.K., Papadopoulos, C., Papadopoulos, I., and Pecora, F. (2017). The CARESSES EU-Japan project: Making assistive robots culturally competent. Italian Forum of Ambient Assisted Living, Springer.
71. Diab, M., Akbari, A., Ud Din, M., and Rosell, J. (2019). PMK—A Knowledge Processing Framework for Autonomous Robotics Perception and Manipulation. Sensors, 19.10.3390/s19051166 ↗
72. Sun, X., Zhang, Y., and Chen, J. (2019). High-Level Smart Decision Making of a Robot Based on Ontology in a Search and Rescue Scenario. Future Internet, 11.10.3390/fi11110230 ↗
73. A Humanoid Social Robot Based Approach for Indoor Environment Quality Monitoring and Well-Being Improvement10.1007/s12369-020-00638-9 ↗
74. An integrated semantic framework for designing context-aware Internet of Robotic Things systems10.3233/ICA-170559 ↗
75. Chang, D.S., Cho, G.H., and Choi, Y.S. (April, January 30). Ontology-based knowledge model for human-robot interactive services. Proceedings of the 35th Annual ACM Symposium on Applied Computing, Brno, Czech Republic.
76. Sadik, A.R., and Urban, B. (2017). An ontology-based approach to enable knowledge representation and reasoning in worker-cobot agile manufacturing. Future Internet, 9.10.3390/fi9040090 ↗
77. Implementation of an ontology-based approach to enable agility in kit building applications10.1142/S1793351X18400019 ↗
78. Gonçalves, P. (2013, January 22–25). Towards an ontology for orthopaedic surgery, application to hip resurfacing. Proceedings of the Hamlyn Symposium on Medical Robotics, London, UK.
79. Diab, M., Akbari, A., and Rosell, J. (2017). An ontology framework for physics-based manipulation planning. Iberian Robotics Conference, Springer.10.1007/978-3-319-70833-1_37 ↗
80. Zhao, J., Gao, J., Zhao, F., and Liu, Y. (2017). A search-and-rescue robot system for remotely sensing the underground coal mine environment. Sensors, 17.10.3390/s17102426 ↗
81. Bujari, A., Calafate, C.T., Cano, J.C., Manzoni, P., Palazzi, C.E., and Ronzani, D. (2018). A location-aware waypoint-based routing protocol for airborne DTNs in search and rescue scenarios. Sensors, 18.10.3390/s18113758 ↗
82. Socially assistive robotics: Human augmentation versus automation10.1126/scirobotics.aam5410 ↗
83. Socially assistive robot for providing recommendations: Comparing a humanoid robot with a mobile application10.1007/s12369-018-0469-4 ↗
84. Investigation of the feasibility of POE methodology for a modern commercial office building10.1016/j.buildenv.2018.07.049 ↗
85. OWL web ontology language overview
86. Wielemaker, J., Schrijvers, T., Triska, M., and Lager, T. (2010). Swi-prolog. arXiv.
87. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., and Ng, A.Y. (2009, January 12). ROS: An open-source Robot Operating System. Proceedings of the ICRA Workshop on Open Source Software, Kobe, Japan.
88. Bordini, R.H., and Hübner, J.F. (2005). BDI agent programming in AgentSpeak using Jason. InInternational Workshop on Computational Logic in Multi-Agent Systems, Springer.10.1007/11750734_9 ↗
89. AgentSpeak (L): BDI agents speak out in a logical computable language
90. Amarilli, F., Amigoni, F., Fugini, M.G., and Zarri, G.P. (2017). A semantic-rich approach to IoT using the generalized world entities paradigm. Manag. Web Things, Elsevier.10.1016/B978-0-12-809764-9.00005-6 ↗
91. Bellifemine, F., Bergenti, F., Caire, G., and Poggi, A. (2005). JADE—A java agent development framework. Multi-Agent Programming, Springer.10.1007/0-387-26350-0_5 ↗
92. Trends in agent communication language10.1111/1467-8640.00184 ↗
93. Towards a core ontology for robotics and automation10.1016/j.robot.2013.04.005 ↗
94. The universaal platform for aal (ambient assisted living)
95. The FF planning system: Fast plan generation through heuristic search10.1613/jair.855 ↗
96. Akbari, A., and Rosell, J. (2016, January 6–9). Task planning using physics-based heuristics on manipulation actions. Proceedings of the 2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA), Berlin, Germany.10.1109/ETFA.2016.7733599 ↗
97. PDDL2. 1: An extension to PDDL for expressing temporal planning domains10.1613/jair.1129 ↗
98. Wang, A.Y., Sable, J.H., and Spackman, K.A. (2002, January 9–13). The SNOMED clinical terms development process: Refinement and analysis of content. Proceedings of the AMIA Symposium, San Antonio, TX, USA.
99. Tenorth, M., and Beetz, M. (2009, January 11–15). KnowRob—Knowledge processing for autonomous personal robots. Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO, USA.10.1109/IROS.2009.5354602 ↗
100. Barattini, P., Vicentini, F., Virk, G.S., and Haidegger, T. (2019). Human-Robot Interaction: Safety, Standardization, and Benchmarking, CRC Press.10.1201/9781315213781 ↗