journal article·2018-03-05Open access

State-of-the-Art Mobile Intelligence: Enabling Robots to Move Like Humans by Estimating Mobility with Artificial Intelligence

Xue-Bo Jin, Ting-Li Su, Jian-Lei Kong, Yu-Ting Bai, Bei-Bei Miao, Chao Dou

Applied Sciences· Volume 8· Issue 3

Abstract

Mobility is a significant robotic task. It is the most important function when robotics is applied to domains such as autonomous cars, home service robots, and autonomous underwater vehicles. Despite extensive research on this topic, robots still suffer from difficulties when moving in complex environments, especially in practical applications. Therefore, the ability to have enough intelligence while moving is a key issue for the success of robots. Researchers have proposed a variety of methods and algorithms, including navigation and tracking. To help readers swiftly understand the recent advances in methodology and algorithms for robot movement, we present this survey, which provides a detailed review of the existing methods of navigation and tracking. In particular, this survey features a relation-based architecture that enables readers to easily grasp the key points of mobile intelligence. We first outline the key problems in robot systems and point out the relationship among robotics, navigation, and tracking. We then illustrate navigation using different sensors and the fusion methods and detail the state estimation and tracking models for target maneuvering. Finally, we address several issues of deep learning as well as the mobile intelligence of robots as suggested future research topics. The contributions of this survey are threefold. First, we review the literature of navigation according to the applied sensors and fusion method. Second, we detail the models for target maneuvering and the existing tracking based on estimation, such as the Kalman filter and its series developed form, according to their model-construction mechanisms: linear, nonlinear, and non-Gaussian white noise. Third, we illustrate the artificial intelligence approach—especially deep learning methods—and discuss its combination with the estimation method.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Robotics and Sensor-Based Localization · Robotic Path Planning Algorithms · Maritime Navigation and Safety

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Computer and Information Engineering, Beijing Technology and Business University, Beijing 100048, China

Beijing Key Laboratory of Big Data Technology for Food Safety, Beijing Technology and Business University, Beijing 100048, China

School of Automation, Beijing Institute of Technology, Beijing 100081, China

Baidu, Inc., Beijing 100085, China

Center of Quality Engineering AVIC China Aero-Polytechnology Establishment, Beijing 100028, China

Funding

NSFC · Beijing Natural Science Foundation · the Key Science and Technology Project of Beijing Municipal Education Commission of China

References (100+)

Show source-provided references
1. Determining optical flow10.1016/0004-3702(81)90024-2 ↗
2. Zhu, A.Z., Atanasov, N., and Daniilidis, K. (June, January 29). Event-based feature tracking with probabilistic data association. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.10.1109/ICRA.2017.7989517 ↗
3. Dan, L., Dai-Hong, J., Rong, B., Jin-Ping, S., Wen-Jing, Z., and Chao, W. (2017, January 14–17). Moving object tracking method based on improved lucas-kanade sparse optical flow algorithm. Proceedings of the 2017 International Smart Cities Conference (ISC2), Wuxi, China.10.1109/ISC2.2017.8090850 ↗
4. Leshed, G., Velden, T., Rieger, O., Kot, B., and Sengers, P. (2008, January 5–10). In-car gps navigation: Engagement with and disengagement from the environment. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy.10.1145/1357054.1357316 ↗
5. Ascher, C., Kessler, C., Wankerl, M., and Trommer, G. (2010, January 15–17). Dual IMU indoor navigation with particle filter based map-matching on a smartphone. Proceedings of the 2010 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Zurich, Switzerland.10.1109/IPIN.2010.5646861 ↗
6. Evolutionary algorithm based offline/online path planner for UAV navigation10.1109/TSMCB.2002.804370 ↗
7. Sensor measurements for Wi-Fi location with emphasis on time-of-arrival ranging10.1109/TMC.2007.1002 ↗
8. Indoor location tracking using RSSI readings from a single Wi-Fi access point10.1007/s11276-006-5064-1 ↗
9. Adams, J.C., Gregorwich, W., Capots, L., and Liccardo, D. (2001, January 10–17). Ultra-wideband for navigation and communications. Proceedings of the 2001 IEEE Aerospace Conference, Big Sky, MT, USA.
10. Speed sensorless vector control of induction motor using extended Kalman filter10.1109/28.315233 ↗
11. Quaternion-based extended Kalman filter for determining orientation by inertial and magnetic sensing10.1109/TBME.2006.875664 ↗
12. Unscented filtering and nonlinear estimation10.1109/JPROC.2003.823141 ↗
13. Cubature kalman filters10.1109/TAC.2009.2019800 ↗
14. Estimating optimal tracking filter performance for manned maneuvering targets10.1109/TAES.1970.310128 ↗
15. Zhou, H., Jing, Z., and Wang, P. (1991). Maneuvering Target Tracking, National Defense Industry Press.
16. Design and comparison of mode-set adaptive IMM algorithms for maneuvering target tracking10.1109/7.745704 ↗
17. Maneuvering target tracking by adaptive statistics model10.1016/S1005-8885(13)60016-3 ↗
18. INS algorithm using quaternion model for low cost IMU10.1016/j.robot.2004.02.001 ↗
19. A Kalman filter-based algorithm for IMU-camera calibration: Observability analysis and performance evaluation10.1109/TRO.2008.2004486 ↗
20. Nistér, D., Naroditsky, O., and Bergen, J. (July, January 27). Visual odometry. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), Washington, DC, USA.
21. Using occupancy grids for mobile robot perception and navigation10.1109/2.30720 ↗
22. Learning metric-topological maps for indoor mobile robot navigation10.1016/S0004-3702(97)00078-7 ↗
23. Vision for mobile robot navigation: A survey10.1109/34.982903 ↗
24. Sensor fusion for mobile robot navigation10.1109/JPROC.1997.554212 ↗
25. Lambooij, M., Fortuin, M., Heynderickx, I., and IJsselsteijn, W. (2009). Visual discomfort and visual fatigue of stereoscopic displays: A review. J. Imaging Sci. Technol., 53.10.2352/J.ImagingSci.Technol.2009.53.3.030201 ↗
26. Kim, Y., and Hwang, D.H. (2016). Vision/INS integrated navigation system for poor vision navigation environments. Sensors, 16.10.3390/s16101672 ↗
27. Flight path planning for unmanned aerial vehicles with landmark-based visual navigation10.1016/j.robot.2013.11.004 ↗
28. Triggs, B., McLauchlan, P.F., Hartley, R.I., and Fitzgibbon, A.W. (1999). Bundle adjustment—A modern synthesis. International Workshop on Vision Algorithms, Springer.10.1007/3-540-44480-7_21 ↗
29. Badino, H., Yamamoto, A., and Kanade, T. (2013, January 2–8). Visual odometry by multi-frame feature integration. Proceedings of the IEEE International Conference on Computer Vision Workshops, Sydney, NSW, Australia.10.1109/ICCVW.2013.37 ↗
30. Tardif, J.P., George, M., Laverne, M., Kelly, A., and Stentz, A. (2010, January 18–22). A new approach to vision-aided inertial navigation. Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, Taiwan.10.1109/IROS.2010.5651059 ↗
31. Bergmann, P., Wang, R., and Cremers, D. (arXiv, 2017). Online Photometric Calibration for Auto Exposure Video for Realtime Visual Odometry and SLAM, arXiv.10.1109/LRA.2017.2777002 ↗
32. Peretroukhin, V., Clement, L., and Kelly, J. (June, January 29). Reducing drift in visual odometry by inferring sun direction using a bayesian convolutional neural network. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.10.1109/ICRA.2017.7989235 ↗
33. Kim, A., and Golnaraghi, M. (2004, January 26–29). A quaternion-based orientation estimation algorithm using an inertial measurement unit. Proceedings of the Position Location and Navigation Symposium, PLANS 2004, Monterey, CA, USA.
34. GPS travelling wave fault locator systems: Investigation into the anomalous measurements related to lightning strikes10.1109/61.517474 ↗
35. Buchli, B., Sutton, F., and Beutel, J. (2012). GPS-equipped wireless sensor network node for high-accuracy positioning applications. European Conference on Wireless Sensor Networks, Springer.10.1007/978-3-642-28169-3_12 ↗
36. Li, B., Zhang, S., and Shen, S. (2016, January 19–21). CSI-based WiFi-inertial state estimation. Proceedings of the 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Baden-Baden, Germany.10.1109/MFI.2016.7849496 ↗
37. Marquez, A., Tank, B., Meghani, S.K., Ahmed, S., and Tepe, K. (May, January 30). Accurate UWB and IMU based indoor localization for autonomous robots. Proceedings of the 2017 IEEE 30th Canadian Conference on Electrical and Computer Engineering (CCECE), Windsor, ON, Canada.10.1109/CCECE.2017.7946751 ↗
38. Lee, J.S., Su, Y.W., and Shen, C.C. (2007, January 5–8). A comparative study of wireless protocols: Bluetooth, UWB, ZigBee, and Wi-Fi. Proceedings of the 33rd Annual Conference of the IEEE Industrial Electronics Society, IECON 2007, Taipei, Taiwan.10.1109/IECON.2007.4460126 ↗
39. Corna, A., Fontana, L., Nacci, A., and Sciuto, D. (2015, January 9–13). Occupancy detection via iBeacon on Android devices for smart building management. Proceedings of the 2015 Design, Automation & Test in Europe Conference & Exhibition, EDA Consortium, Grenoble, France.10.7873/DATE.2015.0753 ↗
40. Lin, X.Y., Ho, T.W., Fang, C.C., Yen, Z.S., Yang, B.J., and Lai, F. (2015, January 25–29). A mobile indoor positioning system based on iBeacon technology. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.
41. He, Z., Cui, B., Zhou, W., and Yokoi, S. (2015, January 22–24). A proposal of interaction system between visitor and collection in museum hall by iBeacon. Proceedings of the 2015 10th International Conference on Computer Science & Education (ICCSE), Cambridge, UK.10.1109/ICCSE.2015.7250283 ↗
42. Koühne, M., and Sieck, J. (2014, January 18–20). Location-based services with iBeacon technology. Proceedings of the 2014 2nd International Conference on Artificial Intelligence, Modelling and Simulation (AIMS), Madrid, Spain.10.1109/AIMS.2014.58 ↗
43. Parallel irregular fusion estimation based on nonlinear filter for indoor RFID tracking system10.1155/2016/1472930 ↗
44. A comprehensive multi-factor analysis on RFID localization capability10.1016/j.aei.2010.05.006 ↗
45. Martin, E., Vinyals, O., Friedland, G., and Bajcsy, R. (2010, January 25–29). Precise indoor localization using smart phones. Proceedings of the 18th ACM international conference on Multimedia. ACM, Firenze, Italy.10.1145/1873951.1874078 ↗
46. WiFi-based indoor positioning10.1109/MCOM.2015.7060497 ↗
47. Ferrera, E., Capitán, J., and Marrón, P.J. (2017). From Fast to Accurate Wireless Map Reconstruction for Human Positioning Systems. Iberian Robotics Conference, Springer.10.1007/978-3-319-70833-1_25 ↗
48. SpotFi: Decimeter Level Localization Using WiFi10.1145/2829988.2787487 ↗
49. An innovative information fusion method with adaptive Kalman filter for integrated INS/GPS navigation of autonomous vehicles10.1016/j.ymssp.2017.07.051 ↗
50. Nguyen, H.D., Nguyen, V.H., and Nguyen, H.V. (2017, January 1–3). Tightly-coupled INS/GPS integration with magnetic aid. Proceedings of the 2017 2nd International Conference on Control and Robotics Engineering (ICCRE), Bangkok, Thailand.10.1109/ICCRE.2017.7935071 ↗
51. Alatise, M.B., and Hancke, G.P. (2017). Pose Estimation of a Mobile Robot Based on Fusion of IMU Data and Vision Data Using an Extended Kalman Filter. Sensors, 17.10.3390/s17102164 ↗
52. Monocular Vision-and IMU-Based System for Prosthesis Pose Estimation During Total Hip Replacement Surgery10.1109/TBCAS.2016.2643626 ↗
53. Malyavej, V., Kumkeaw, W., and Aorpimai, M. (2013, January 15–17). Indoor robot localization by RSSI/IMU sensor fusion. Proceedings of the 2013 10th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), Krabi, Thailand.10.1109/ECTICon.2013.6559517 ↗
54. Zhang, P., Gu, J., Milios, E.E., and Huynh, P. (August, January 29). Navigation with IMU/GPS/digital compass with unscented Kalman filter. Proceedings of the 2005 IEEE International Conference Mechatronics and Automation, Niagara Falls, ON, Canada.
55. Ma, S., Zhang, Y., Xu, Y., Wang, B., Cheng, J., and Zhao, Q. (2016, January 15–17). Indoor robot navigation by coupling IMU, UWB, and encode. Proceedings of the 2016 10th International Conference on Software, Knowledge, Information Management & Applications (SKIMA), Chengdu, China.10.1109/SKIMA.2016.7916260 ↗
56. Data Fusion for Indoor Mobile Robot Positioning Based on Tightly Coupled INS/UWB10.1017/S0373463317000194 ↗
57. Accurate pedestrian indoor navigation by tightly coupling foot-mounted IMU and RFID measurements10.1109/TIM.2011.2159317 ↗
58. Jiménez, A.R., Seco, F., Zampella, F., Prieto, J.C., and Guevara, J. (2012). Indoor localization of persons in aal scenarios using an inertial measurement unit (IMU) and the signal strength (SS) from RFID tags. International Competition on Evaluating AAL Systems through Competitive Benchmarking, Springer.10.1007/978-3-642-37419-7_4 ↗
59. GPS/IMU data fusion using multisensor Kalman filtering: Introduction of contextual aspects10.1016/j.inffus.2004.07.002 ↗
60. Rios, J.A., and White, E. (2002). Fusion Filter Algorithm Enhancements for a MEMS GPS/IMU, Crossbow Technology Inc.
61. Estimating vehicle state by GPS/IMU fusion with vehicle dynamics10.1007/s10846-013-9960-1 ↗
62. Werries, A., and Dolan, J.M. (2016). Adaptive Kalman Filtering Methods for Low-Cost GPS/INS Localization for Autonomous Vehicles, Research Showcase CMU. Available online: http://http://repository.cmu.edu/robotics/.
63. Applying Time-Differenced Carrier Phase in Nondifferential GPS/IMU Tightly Coupled Navigation Systems to Improve the Positioning Performance10.1109/TVT.2016.2558206 ↗
64. Caltagirone, L., Bellone, M., Svensson, L., and Wahde, M. (2017). LIDAR-based Driving Path Generation Using Fully Convolutional Neural Networks, Cornell University arXiv Institution.10.1109/IVS.2017.7995848 ↗
65. Non-Linear Autoregressive Delay-Dependent INS/GPS Navigation System Using Neural Networks10.1109/JSEN.2016.2642040 ↗
66. Sensor fusion of camera, GPS and IMU using fuzzy adaptive multiple motion models
67. Huang, G., Eckenhoff, K., and Leonard, J. (2018). Optimal-state-constraint EKF for visual-inertial navigation. Robotics Research, Springer.10.1007/978-3-319-51532-8_8 ↗
68. A Vision/Inertia Integrated Positioning Method Using Position and Orientation Matching10.1155/2017/6835456 ↗
69. Monocular visual-IMU odometry using multi-channel image patch exemplars10.1007/s11042-016-3927-8 ↗
70. Ascani, A., Frontoni, E., Mancini, A., and Zingaretti, P. (2008, January 22–26). Feature group matching for appearance-based localization. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2008, Nice, France.10.1109/IROS.2008.4651023 ↗
71. Audi, A., Pierrot-Deseilligny, M., Meynard, C., and Thom, C. (2017). Implementation of an IMU Aided Image Stacking Algorithm in a Digital Camera for Unmanned Aerial Vehicles. Sensors, 17.10.3390/s17071646 ↗
72. Kneip, L., Chli, M., and Siegwart, R.Y. (2011). Robust real-time visual odometry with a single camera and an IMU. Proceedings of the British Machine Vision Conference 2011, British Machine Vision Association.10.5244/C.25.16 ↗
73. Spaenlehauer, A., Fremont, V., Sekercioglu, Y.A., and Fantoni, I. (2017). A Loosely-Coupled Approach for Metric Scale Estimation in Monocular Vision-Inertial Systems, Cornell University arXiv Institution.10.1109/MFI.2017.8170419 ↗
74. An algorithm for tracking multiple targets10.1109/TAC.1979.1102177 ↗
75. A new approach to linear filtering and prediction problems10.1115/1.3662552 ↗
76. Grewal, M.S. (2011). Kalman filtering. International Encyclopedia of Statistical Science, Springer.10.1007/978-3-642-04898-2_321 ↗
77. Sinharay, A., Pal, A., and Bhowmick, B. (April, January 30). A kalman filter based approach to de-noise the stereo vision based pedestrian position estimation. Proceedings of the 2011 UkSim 13th International Conference on Computer Modelling and Simulation (UKSim), Cambridge, UK.10.1109/UKSIM.2011.30 ↗
78. Asymptotic behavior of the extended Kalman filter as a parameter estimator for linear systems10.1109/TAC.1979.1101943 ↗
79. Structural identification by extended Kalman filter10.1061/(ASCE)0733-9399(1984)110:12(1757) ↗
80. Design and implementation of an extended Kalman filter for the state estimation of a permanent magnet synchronous motor10.1109/63.85891 ↗
81. Marins, J.L., Yun, X., Bachmann, E.R., McGhee, R.B., and Zyda, M.J. (November, January 29). An extended Kalman filter for quaternion-based orientation estimation using MARG sensors. Proceedings of the 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems, Maui, HI, USA.
82. Yang, S., and Baum, M. (2017, January 5–9). Extended Kalman filter for extended object tracking. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952985 ↗
83. Julier, S.J. (2002, January 8–10). The scaled unscented transformation. Proceedings of the 2002 American Control Conference, Anchorage, AK, USA.10.1109/ACC.2002.1025369 ↗
84. Performance comparison of EKF/UKF/CKF for the tracking of ballistic target
85. Ding, Z., and Balaji, B. (2012, January 22–25). Comparison of the unscented and cubature Kalman filters for radar tracking applications. Proceedings of the IET International Conference on Radar Systems, Glasgow, UK.
86. Concert Assessment of Unscented and Cubature Kalman Filters for Target Tracking
87. Pesonen, H., and Piché, R. (2010, January 11–12). Cubature-based Kalman filters for positioning. Proceedings of the 2010 7th Workshop on Positioning Navigation and Communication (WPNC), Dresden, Germany.10.1109/WPNC.2010.5653829 ↗
88. Transformed unscented Kalman filter10.1109/TAC.2012.2204830 ↗
89. Stochastic integration filter10.1109/TAC.2013.2258494 ↗
90. Improved particle filter for nonlinear problems10.1049/ip-rsn:19990255 ↗
91. Van Der Merwe, R., Doucet, A., De Freitas, N., and Wan, E.A. (2001, January 3–8). The unscented particle filter. Proceedings of the Advances in Neural Information Processing Systems 14 (NIPS 2001), Vancouver, BC, Canada.
92. A sequential particle filter method for static models10.1093/biomet/89.3.539 ↗
93. An adaptive color-based particle filter10.1016/S0262-8856(02)00129-4 ↗
94. A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking10.1109/78.978374 ↗
95. Efficient Particle Swarm Optimized Particle Filter Based Improved Multiple Model Tracking Algorithm10.1111/coin.12084 ↗
96. A proposal for the dartmouth summer research project on artificial intelligence, 31 August 1955
98. Frank, R. (1957). The Perceptron a Perceiving and Recognizing Automaton, Cornell Aeronautical Laboratory. Tech. Rep.
99. Crevier, D. (1993). AI: The Tumultuous History of the Search for Artificial Intelligence, Basic Books.
100. A fast learning algorithm for deep belief nets10.1162/neco.2006.18.7.1527 ↗