journal article·2020-01-09Open access

Semantic Information for Robot Navigation: A Survey

Jonathan Crespo, Jose Carlos Castillo, Oscar Martinez Mozos, Ramon Barber

Applied Sciences· Volume 10· Issue 2

Abstract

There is a growing trend in robotics for implementing behavioural mechanisms based on human psychology, such as the processes associated with thinking. Semantic knowledge has opened new paths in robot navigation, allowing a higher level of abstraction in the representation of information. In contrast with the early years, when navigation relied on geometric navigators that interpreted the environment as a series of accessible areas or later developments that led to the use of graph theory, semantic information has moved robot navigation one step further. This work presents a survey on the concepts, methodologies and techniques that allow including semantic information in robot navigation systems. The techniques involved have to deal with a range of tasks from modelling the environment and building a semantic map, to including methods to learn new concepts and the representation of the knowledge acquired, in many cases through interaction with users. As understanding the environment is essential to achieve high-level navigation, this paper reviews techniques for acquisition of semantic information, paying attention to the two main groups: human-assisted and autonomous techniques. Some state-of-the-art semantic knowledge representations are also studied, including ontologies, cognitive maps and semantic maps. All of this leads to a recent concept, semantic navigation, which integrates the previous topics to generate high-level navigation systems able to deal with real-world complex situations.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Semantic Web and Ontologies · Robotics and Automated Systems · AI-based Problem Solving and Planning

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Higher Technical School of Computer Engineering, UniversityRey Juan Carlos, 28933 Móstoles, Spain

Department of Systems Engineering and Automation, University Carlos III of Madrid, 28911 Leganés, Spain

Centre for Applied Autonomous Sensor Systems, Örebro University, 70182 Örebro, Sweden

References (100+)

Show source-provided references
1. Semantic maps from multiple visual cues10.1016/j.eswa.2016.10.014 ↗
2. Semantic mapping for mobile robotics tasks: A survey10.1016/j.robot.2014.12.006 ↗
3. Kollar, T., and Roy, N. (2009, January 12–17). Utilizing object-object and object-scene context when planning to find things. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Kobe, Japan.10.1109/ROBOT.2009.5152831 ↗
4. Arras, K. (2003). Featur-Based Robot Navigation in Know and Unknow Environments. [Ph.D. Thesis, Swiss Federal Institute of Technology Lausanne (EPFL)].
5. Chatila, R., and Laumond, J. (1985, January 25–28). Position referencing and consistent world modeling for mobile robots. Proceedings of the IEEE International Conference on Robotics and Automation, St. Louis, MO, USA.
6. Jiang, G., Yin, L., Jin, S., Tian, C., Ma, X., and Ou, Y. (2019). A Simultaneous Localization and Mapping (SLAM) Framework for 2.5D Map Building Based on Low-Cost LiDAR and Vision Fusion. Appl. Sci., 9.10.3390/app9102105 ↗
7. Topological Simultaneous Localization And Mapping (SLAM): Toward exact localization without explicit localization10.1109/70.928558 ↗
8. Tapus, A. (2005). Topological SLAM-Simultaneous Localization And Mapping with Fingerprints of Places. [Ph.D. Thesis, Swiss Federal Institute of Technology Lausanne (EPFL)].
9. Wang, F., Liu, Y., Xiao, L., Wu, C., and Chu, H. (2019). Topological Map Construction Based on Region Dynamic Growing and Map Representation Method. Appl. Sci., 9.10.3390/app9050816 ↗
10. Learning metric-topological maps for indoor mobile robot navigation10.1016/S0004-3702(97)00078-7 ↗
11. Hybrid simultaneous localization and map building: A natural integration of topological and metric, Robotics and Autonomous Systems10.1016/S0921-8890(03)00006-X ↗
12. Toward Principles for the Design of Ontologies Used for Knowledge Sharing10.1006/ijhc.1995.1081 ↗
13. Galindo, C., Saffiotti, A., Coradeschi, S., Buschka, P., Fernández-Madrigal, J.A., and González, J. (2005, January 2–6). Multi-hierarchical semantic maps for mobile robotics. Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AL, Canada.10.1109/IROS.2005.1545511 ↗
14. Fernández-Rodicio, E., Castro-González, Á., Castillo, J.C., Alonso-Martin, F., and Salichs, M.A. (2018). Composable Multimodal Dialogues Based on Communicative Acts. Social Robotics, Springer International Publishing.10.1007/978-3-030-05204-1_14 ↗
15. Conceptual spatial representations for indoor mobile robots10.1016/j.robot.2008.03.007 ↗
16. Nieto-Granda, C., Rogers, J.G., Trevor, A.J.B., and Christensen., H.I. (2010, January 18–22). Semantic Map Partitioning in Indoor Environments using Regional Analysis. Proceedings of the IEEEl/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan.10.1109/IROS.2010.5650575 ↗
17. Pronobis, A., and Jensfelt., P. (2012, January 14–18). Large-scale semantic mapping and reasoning with heterogeneous modalities. Proceedings of the 2012 IEEE International Conference on Robotics and Automation (ICRA’12), St. Paul, MN, USA.10.1109/ICRA.2012.6224637 ↗
18. Peltason, J., Siepmann, F.H.K., Spexard, T.P., Wrede, B., Hanheide, M., and Topp, E.A. (2009, January 12–17). Mixed-initiative in human augmented mapping. Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan.10.1109/ROBOT.2009.5152683 ↗
19. Relational Model for Robotic Semantic Navigation in Indoor Environments10.1007/s10846-017-0469-x ↗
20. Barber, R., Crespo, J., Gomez, C., Hernandez, A.C., and Galli, M. (2018). Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation. Applications of Mobile Robots, IntechOpen.10.5772/intechopen.79842 ↗
21. Kruijff, G.J.M., Zender, H., Jensfelt, P., and Christensen, H.I. (2006, January 2–3). Clarification Dialogues in Human-augmented Mapping. Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-robot Interaction, HRI ’06, Salt Lake City, UT, USA.10.1145/1121241.1121290 ↗
22. Hemachandra, S., Kollar, T., Roy, N., and Teller, S.J. (2011, January 9–13). Following and interpreting narrated guided tours. Proceedings of the Robotics and Automation (ICRA), Shanghai, China.10.1109/ICRA.2011.5980209 ↗
23. Living with robots: Interactive environmental knowledge acquisition10.1016/j.robot.2015.11.001 ↗
24. Goerke, N., and Braun, S. (2009, January 15). Building Semantic Annotated Maps by Mobile Robots. Proceedings of the Towards Autonomous Robotic Systems, Londonderry, UK.
25. Brunskill, E., Kollar, T., and Roy, N. (November, January 29). Topological mapping using spectral clustering and classification. Proceedings of the IEEE/RSJ Conference on Robots and Systems, San Diego, CA, USA.
26. Friedman, S., Pasula, H., and Fox, D. (2007, January 6–12). Voronoi random fields: Extracting the topological structure of indoor environments via place labeling. Proceedings of the International Joint Conference on Artificial Intelligence, Hyderabad, India.
27. Wu, J., Christensen, H.I., and Rehg, J.M. (2009, January 10–15). Visual place categorization: Problem, dataset, and algorithm. Proceedings of the IROS’09, St. Louis, MO, USA.10.1109/IROS.2009.5354164 ↗
28. Tian, B., Shim, V.A., Yuan, M., Srinivasan, C., Tang, H., and Li, H. (2013, January 3–7). RGB-D based cognitive map building and navigation. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan.
29. Crespo, J., Gómez, C., Hernández, A., and Barber, R. (2017). A Semantic Labeling of the Environment Based on What People Do. Sensors, 17.10.3390/s17020260 ↗
30. Distinctive image features from scale-invariant keypoints10.1023/B:VISI.0000029664.99615.94 ↗
31. Object detection and mapping for service robot tasks10.1017/S0263574706003237 ↗
32. Ekvall, S., and Kragic, D. (2005, January 2–6). Receptive field cooccurrence histograms for object detection. Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AL, Canada.10.1109/IROS.2005.1545588 ↗
33. López, D. (2007). Combining Object Recognition and Metric Mapping for Spatial Modeling with Mobile Robots. [Master’s Thesis, Royal Institute of Technology].
34. Aydemir, A., Göbelbecker, M., Pronobis, A., Sjöö, K., and Jensfelt., P. (2011, January 7–9). Plan-based object search and exploration using semantic spatial knowledge in the real world. Proceedings of the 5th European Conference on Mobile Robots (ECMR11), Örebro, Sweden.
35. Garvey, T.D. (1976). Perceptual Strategies for Purposive Vision, SRI International.
36. Furniture models learned from the WWW—Using web catalogs to locate and categorize unknown furniture pieces in 3D laser scans10.1109/MRA.2011.940996 ↗
37. Joho, D., and Burgard, W. (2010, January 3–7). Searching for Objects: Combining Multiple Cues to Object Locations Using a Maximum Entropy Model. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Anchorage, AK, USA.10.1109/ROBOT.2010.5509285 ↗
38. Object Detection Techniques Applied on Mobile Robot Semantic Navigation10.3390/s140406734 ↗
39. Blodow, N., Goron, L.C., and Marton, Z.C. (2011, January 25–30). Autonomous semantic mapping for robots performing everyday manipulation tasks in kitchen environments. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, CA, USA.10.1109/IROS.2011.6094665 ↗
40. Vasudevan, S., and Siegwart, R. (November, January 29). A Bayesian approach to Conceptualization and Place Classification: Incorporating Spatial Relationships (distances) between Objects towards inferring concepts. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Diego, CA, USA.
41. Vasudevan, S., Harati, A., and Siegwart, R. (2007, January 19–21). A Bayesian Conceptualization of Space for Mobile Robots: Using the Number of Occurrences of Objects to Infer Concepts. Proceedings of the European Conference on Mobile Robotics (ECMR), Freiburg, Germany.10.1109/IROS.2007.4399099 ↗
42. Miyamoto, R., Adachi, M., Nakamura, Y., Nakajima, T., Ishida, H., and Kobayashi, S. (2019, January 23–26). Accuracy improvement of semantic segmentation using appropriate datasets for robot navigation. Proceedings of the 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT), Paris, France.10.1109/CoDIT.2019.8820616 ↗
43. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid scene parsing network. Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
44. Mousavian, A., Toshev, A., Fišer, M., Košecká, J., Wahid, A., and Davidson, J. (2019, January 20–24). Visual representations for semantic target driven navigation. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.10.1109/ICRA.2019.8793493 ↗
45. Real-time people counting for indoor scenes10.1016/j.sigpro.2015.10.036 ↗
46. Leg Detection and Tracking for a Mobile Robot and Based on a Laser Device, Supervised Learning and Particle Filtering
47. Monroy, J., Ruiz-Sarmiento, J., Moreno, F., Galindo, C., and Gonzalez-Jimenez, J. (2019). Olfaction, Vision, and Semantics for Mobile Robots. Results of the IRO Project. Sensors, 19.10.3390/s19163488 ↗
48. Learning spatially semantic representations for cognitive robot navigation10.1016/j.robot.2013.07.008 ↗
49. Drouilly, R., Rives, P., and Morisset, B. (2015, January 26–30). Semantic Representation For Navigation In Large-Scale Environments. Proceedings of the IEEE International Conference on Robotics and Automation, ICRA’15, Seattle, WA, USA.10.1109/ICRA.2015.7139314 ↗
50. Cleveland, J., Thakur, D., Dames, P., Phillips, C., Kientz, T., Daniilidis, K., Bergstrom, J., and Kumar, V. (2015, January 24–28). An automated system for semantic object labeling with soft object recognition and dynamic programming segmentation. Proceedings of the 2015 IEEE International Conference on Automation Science and Engineering (CASE), Gothenburg, Sweden.10.1109/CoASE.2015.7294159 ↗
51. Rituerto, J., Murillo, A.C., and Košecka, J. (2011, January 25–30). Label propagation in videos indoors with an incremental non-parametric model update. Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, CA, USA.10.1109/IROS.2011.6048643 ↗
52. Shi, L., Kodagoda, S., and Dissanayake, G. (2010, January 7–10). Multi-class classification for semantic labeling of places. Proceedings of the 2010 11th International Conference on Control Automation Robotics & Vision (ICARCV), Singapore.10.1109/ICARCV.2010.5707856 ↗
53. SIFT—The scale invariant feature transform
54. Polastro, R., Corrêa, F., Cozman, F., and Okamoto, J. (2010). Semantic mapping with a probabilistic description logic. Advances in Artificial Intelligence—SBIA 2010, Springer.10.1007/978-3-642-16138-4_7 ↗
55. MiNERVA: Toposemantic Navigation Model based on Visual Information for Indoor Enviroments10.1016/j.ifacol.2019.08.046 ↗
56. Visual semantic navigation based on deep learning for indoor mobile robots
57. Supervised semantic labeling of places using information extracted from sensor data10.1016/j.robot.2006.12.003 ↗
58. Sousa, P., Araujo, R., and Nunes, U. (2007, January 4–7). Real-time labeling of places using support vector machines. Proceedings of the ISIE 2007—IEEE International Symposium on Industrial Electronics, Vigo, Spain.10.1109/ISIE.2007.4374918 ↗
59. Multi-modal Semantic Place Classification10.1177/0278364909356483 ↗
60. Mozos, O.M., Stachniss, C., and Burgard, W. (2005, January 18–22). Supervised learning of places from range data using adaboost. Proceedings of the 2005 IEEE International Conference on Robotics and Automation—ICRA 2005, Barcelona, Spain.
61. Rottmann, A., Mozos, O.M., Stachniss, C., and Burgard, W. (2005, January 9–13). Semantic Place Classification of Indoor Environments With Mobile Robots using Boosting. Proceedings of the National Conference on Artificial Intelligence (AAAI), Pittsburgh, PA, USA.
62. Recent trends in social aware robot navigation: A survey10.1016/j.robot.2017.03.002 ↗
63. Müller, A.C., and Behnke, S. (June, January 31). Learning depth-sensitive conditional random fields for semantic segmentation of RGB-D images. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.10.1109/ICRA.2014.6907778 ↗
64. Wolf, D., Prankl, J., and Vincze, M. (2015, January 26–30). Fast semantic segmentation of 3D point clouds using a dense CRF with learned parameters. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.10.1109/ICRA.2015.7139875 ↗
65. Gutiérrez-Gómez, D., Mayol-Cuevas, W., and Guerrero, J.J. (2015, January 26–30). What should I landmark? Entropy of normals in depth juts for place recognition in changing environments using RGB-D data. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.10.1109/ICRA.2015.7139963 ↗
66. Hermans, A., Floros, G., and Leibe, B. (June, January 31). Dense 3D semantic mapping of indoor scenes from RGB-D images. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.10.1109/ICRA.2014.6907236 ↗
67. PLISS: Labeling places using online changepoint detection10.1007/s10514-012-9273-4 ↗
68. Bags of Binary Words for Fast Place Recognition in Image Sequences10.1109/TRO.2012.2197158 ↗
69. Maffra, F., Chen, Z., and Chli, M. (2018, January 21–25). Tolerant place recognition combining 2D and 3D information for UAV navigation. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.10.1109/ICRA.2018.8460786 ↗
70. Lu, D.V., Hershberger, D., and Smart, W.D. (2014, January 14–18). Layered costmaps for context-sensitive navigation. Proceedings of the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA.10.1109/IROS.2014.6942636 ↗
71. Wong, L.L., Kaelbling, L.P., and Lozano-Perez, T. (June, January 31). Not Seeing is Also Believing: Combining Object and Metric Spatial Information. Proceedings of the IEEE Conference on Robotics and Automation (ICRA), Hong Kong, China.
72. Zhao, Z., and Chen, X. (2014, January 14–18). Semantic Mapping for Object Category and Structural Class. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Chicago, IL, USA.10.1109/IROS.2014.6942638 ↗
73. Deeken, H., Wiemann, T., Lingemann, K., and Hertzberg, J. (2015, January 2–4). SEMAP—A semantic environment mapping framework. Proceedings of the 2015 European Conference on Mobile Robots (ECMR), Lincoln, UK.10.1109/ECMR.2015.7324176 ↗
74. Walter, M.R., Hemachandra, S., Homberg, B., Tellex, S., and Teller, S. (2013, January 24–28). Learning Semantic Maps from Natural Language Descriptions. Proceedings of the 2013 Robotics: Science and Systems IX Conference, Berlin, Germany.10.15607/RSS.2013.IX.004 ↗
75. Hemachandra, S., and Walter, M.R. (October, January 28). Information-theoretic dialog to improve spatial-semantic representations. Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.10.1109/IROS.2015.7354097 ↗
76. Ko, D.W., Yi, C., and Suh, I.H. (2013, January 3–7). Semantic mapping and navigation: A Bayesian approach. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.
77. Patel-Schneider, P., Abrtahams, M., Alperin, L., McGuinness, D., and Borgida, A. (1996). NeoClassic Reference Manual: Version 1.0, AT&T Labs Research, Artificial Intelligence Principles Research Department. Technical Report.
78. Crespo, J. (2017). Arquitectura y Diseno de un Sistema Completo de Navegacion Semantica. Descripcion de su Ontologia y Gestion de Conocimiento. [Ph.D. Thesis, Universidad Carlos III de Madrid].
79. Building Multiversal Semantic Maps for Mobile Robot Operation10.1016/j.knosys.2016.12.016 ↗
80. Ontologies: Principles, methods and applications10.1017/S0269888900007797 ↗
81. Robot task planning using semantic Maps10.1016/j.robot.2008.08.007 ↗
82. Inferring robot goals from violations of semantic knowledge10.1016/j.robot.2012.12.007 ↗
83. Towards semantic maps for mobile robots10.1016/j.robot.2008.08.001 ↗
84. Tenorth, M., Kunze, L., Jain, D., and Beetz, M. (2010, January 6–8). KNOWROB-MAP—Knowledge-Linked Semantic Object Maps. Proceedings of the IEEE-RAS International Conference on Humanoid Robots, Nashville, TN, USA.10.1109/ICHR.2010.5686350 ↗
85. Improved techniques for grid mapping with Rao-Blackwellized particle filters10.1109/TRO.2006.889486 ↗
86. Beeson, P., Jong, N.K., and Kuipers, B. (2005, January 18–22). Towards autonomous topological place detection using the extended voronoi graph. Proceedings of the IEEE International Conference on Robotics and Automation, Barcelona, Spain.
87. Towards a Core Ontology for Robotics and Automation10.1016/j.robot.2013.04.005 ↗
88. Knowledge engineering: Principles and methods10.1016/S0169-023X(97)00056-6 ↗
89. Guarino, N. (1998, January 6–8). Formal Ontology and Information Systems. Proceedings of the First International Conference On Formal Ontology In Information Systems, Trento, Italy.
90. Gómez-Pérez, A., Fernández-López, M., and Corcho, O. (2007). Ontological Engineering: With Examples from the Areas of Knowledge Management, e-Commerce and the Semantic Web. (Advanced Information and Knowledge Processing), Springer.
91. Cognitive maps for mobile robots—An object based approach10.1016/j.robot.2006.12.008 ↗
92. Mapping a suburb with a single camera using a biologically inspired slam system10.1109/TRO.2008.2004520 ↗
93. Shim, V.A., Tian, B., Yuan, M., Tang, H., and Li, H. (2014, January 14–18). Direction-driven navigation using cognitive map for mobile robots. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Chicago, IL, USA.10.1109/IROS.2014.6942923 ↗
94. Rebai, K., Azouaoui, O., and Achour, N. (2012, January 7–12). Bio-inspired visual memory for robot cognitive map building and scene recognition. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal.10.1109/IROS.2012.6385493 ↗
95. Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System10.1016/0893-6080(91)90056-B ↗
96. Ulrich, I., and Nourbakhsh, I. (2000, January 24–28). Appearance-Based Place Recognition for Topological Localization. Proceedings of the IEEE International Conference on Robotics and Automation, San Francisco, CA, USA.
97. Biologically inspired mobile robot vision localization10.1109/TRO.2009.2022424 ↗
98. Spatial semantic hybrid map building and application of mobile service robot10.1016/j.robot.2013.01.001 ↗
99. Subjective local maps for hybrid metric-topological SLAM10.1016/j.robot.2008.02.002 ↗
100. Li, X., Wang, D., Ao, H., Belaroussi, R., and Gruyer, D. (2019). Fast 3D Semantic Mapping in Road Scenes. Appl. Sci., 9.10.20944/preprints201901.0009.v1 ↗