A CNN Regression Approach to Mobile Robot Localization Using Omnidirectional Images
Source abstract
Abstract
Understanding the environment is an essential ability for robots to be autonomous. In this sense, Convolutional Neural Networks (CNNs) can provide holistic descriptors of a scene. These descriptors have proved to be robust in dynamic environments. The aim of this paper is to perform hierarchical localization of a mobile robot in an indoor environment by means of a CNN. Omnidirectional images are used as the input of the CNN. Experiments include a classification study in which the CNN is trained so that the robot is able to find out the room where it is located. Additionally, a transfer learning technique transforms the original CNN into a regression CNN which is able to estimate the coordinates of the position of the robot in a specific room. Regarding classification, the room retrieval task is performed with considerable success. As for the regression stage, when it is performed along with an approach based on splitting rooms, it also provides relatively accurate results.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotics and Sensor-Based Localization · Advanced Image and Video Retrieval Techniques · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Systems Engineering and Automation, Miguel Hernández University, 03202 Elche, Spain
References (48)
Show source-provided references
1. Hamid, M., Adom, A., Rahim, N., and Rahiman, M. (2009, January 6–8). Navigation of mobile robot using Global Positioning System (GPS) and obstacle avoidance system with commanded loop daisy chaining application method. Proceedings of the 2009 5th International Colloquium on Signal Processing Its Applications, Kuala Lumpur, Malaysia.10.1109/CSPA.2009.5069211 ↗
2. Markom, M.A., Adom, A.H., Tan, E.S.M.M., Shukor, S.A.A., Rahim, N.A., and Shakaff, A.Y.M. (2015, January 18–20). A mapping mobile robot using RP Lidar scanner. Proceedings of the 2015 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), Langkawi, Malaysia.10.1109/IRIS.2015.7451592 ↗
3. Mobile robot map building from time-of-flight camera10.1016/j.eswa.2012.02.006 ↗
4. Jiang, G., Yin, L., Jin, S., Tian, C., Ma, X., and Ou, Y. (2019). A Simultaneous Localization and Mapping (SLAM) Framework for 2.5D Map Building Based on Low-Cost LiDAR and Vision Fusion. Appl. Sci., 9.10.3390/app9102105 ↗
5. Ballesta, M., Gil, A., Reinoso, O., and Payá, L. (2011). Building Visual Maps with a Team of Mobile Robots, INTECHOpen. Chapter 6.10.5772/13122 ↗
6. Se, S., Lowe, D., and Little, J. (2001, January 21–26). Vision-based mobile robot localization and mapping using scale-invariant features. Proceedings of the 2001 ICRA. IEEE International Conference on Robotics and Automation, Seoul, Korea.
7. Estimation of Visual Maps with a Robot Network Equipped with Vision Sensors10.3390/s100505209 ↗
8. Ali, S., Al Mamun, S., Fukuda, H., Lam, A., Kobayashi, Y., and Kuno, Y. (2018). Smart Robotic Wheelchair for Bus Boarding Using CNN Combined with Hough Transforms, Springer. In International Conference on Intelligent Computing.10.1007/978-3-319-95957-3_18 ↗
9. A Survey of the Recent Architectures of Deep Convolutional Neural Networks10.1007/s10462-020-09825-6 ↗
10. Andreasson, H. (2008). Local Visual Feature Based Localisation and Mapping by Mobile Robots. [Ph.D. Thesis, Örebro University]. ISBN 978-91-7668-614-0.
11. Ravankar, A.A., Ravankar, A., Emaru, T., and Kobayashi, Y. (2020). Multi-Robot Mapping and Navigation Using Topological Features. Proceedings, 42.10.3390/ecsa-6-06580 ↗
12. Object Recognition from Local Scale-Invariant Features
13. Speeded-Up Robust Features (SURF)10.1016/j.cviu.2007.09.014 ↗
14. Holistic Descriptors of Omnidirectional Color Images and Their Performance in Estimation of Position and Orientation10.1109/ACCESS.2020.2990996 ↗
15. Principal Component Analysis10.1002/wics.101 ↗
16. Huang, G., Liu, Z., and Weinberger, K. (2016). Densely Connected Convolutional Networks. arXiv.10.1109/CVPR.2017.243 ↗
17. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
18. Imagenet classification with deep convolutional neural networks
19. Deep Learning10.1038/nature14539 ↗
20. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
21. Taoufiq, S., Nagy, B., and Benedek, C. (2020). HierarchyNet: Hierarchical CNN-Based Urban Building Classification. Remote Sens., 12.10.3390/rs12223794 ↗
22. Localization of Moving Robots By Using Omnidirectional Camera in State Space Framework
23. Huei-Yung, L., and Chien-Hsing, H. (2021). Mobile Robot Self-Localization Using Omnidirectional Vision with Feature Matching from Real and Virtual Spaces. Appl. Sci., 11.10.3390/app11083360 ↗
24. Mobile Robot Localization through Unsupervised Learning using Omnidirectional Images
25. Mapping and localization module in a mobile robot for insulating building crawl spaces10.1016/j.autcon.2017.11.007 ↗
26. Backpropagation Applied to Handwritten Zip Code Recognition10.1162/neco.1989.1.4.541 ↗
27. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
28. He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.10.1109/CVPR.2016.90 ↗
29. Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
30. Szegedy, C., Ioffe, S., and Vanhoucke, V. (2016). Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. arXiv.10.1609/aaai.v31i1.11231 ↗
31. Da Silva, S.P.P., da Nòbrega, R.V.M., Medeiros, A.G., Marinho, L.B., Almeida, J.S., and Filho, P.P.R. (2018, January 8–13). Localization of Mobile Robots with Topological Maps and Classification with Reject Option using Convolutional Neural Networks in Omnidirectional Images. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.10.1109/IJCNN.2018.8489328 ↗
32. Kim, K.W., Hong, H.G., Nam, G.P., and Park, K.R. (2017). A Study of Deep CNN-Based Classification of Open and Closed Eyes Using a Visible Light Camera Sensor. Sensors, 17.10.3390/s17071534 ↗
33. Hierarchical Localization in Topological Models Under Varying Illumination Using Holistic Visual Descriptors10.1109/ACCESS.2019.2910581 ↗
34. Kanezaki, A., Matsushita, Y., and Nishida, Y. (2018, January 18–23). Rotationnet: Joint object categorization and pose estimation using multiviews from unsupervised viewpoints. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00526 ↗
35. Sünderhauf, N., Shirazi, S., Dayoub, F., Upcroft, B., and Milford, M. (October, January 28). On the performance of ConvNet features for place recognition. Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.
36. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
37. Girshick, R. (2015, January 13–16). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
38. Faster r-cnn: Towards real-time object detection with region proposal networks
39. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask r-cnn. Proceedings of the International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.322 ↗
40. Kunii, Y., Kovacs, G., and Hoshi, N. (2017, January 19–21). Mobile robot navigation in natural environments using robust object tracking. Proceedings of the IEEE 26th International Symposium on Industrial Electronics (ISIE), Edinburgh, UK.10.1109/ISIE.2017.8001512 ↗
41. Using omnidirectional vision to create a model of the environment: A comparative evaluation of global-appearance descriptors
42. Sinha, H., Patrikar, J., Dhekane, E.G., Pandey, G., and Kothari, M. (2018, January 27–30). Convolutional Neural Network Based Sensors for Mobile Robot Relocalization. Proceedings of the 23rd International Conference on Methods Models in Automation Robotics (MMAR), Miedzyzdroje, Poland.10.1109/MMAR.2018.8485921 ↗
43. Payá, L., Peidró, A., Amorós, F., Valiente, D., and Reinoso, O. (2018). Modeling Environments Hierarchically with Omnidirectional Imaging and Global-Appearance Descriptors. Remote Sens., 10.10.3390/rs10040522 ↗
44. Xu, S., Chou, W., and Dong, H. (2019). A robust indoor localization system integrating visual localization aided by CNN-based image retrieval with Monte Carlo localization. Sensors, 19.10.3390/s19020249 ↗
45. Chaves, D., Ruiz-Sarmiento, J., Petkov, N., and Gonzalez-Jimenez, J. (2019, January 12–14). Integration of CNN into a Robotic Architecture to Build Semantic Maps of Indoor Environments. Proceedings of the Advances in Computational Intelligence, Gran Canaria, Spain.10.1007/978-3-030-20518-8_27 ↗
46. COLD: COsy Localization Database10.1177/0278364909103912 ↗
47. (2021, July 21). COsy Localization Database. Available online: https://www.cas.kth.se/COLD/cold-freiburg.html.
48. Cebollada, S., Payá, L., Mayol, W., and Reinoso, O. (2019). Evaluation of Clustering Methods in Compression of Topological Models and Visual Place Recognition Using Global Appearance Descriptors. Appl. Sci., 9.10.3390/app9030377 ↗