Expanding Ground Vehicle Autonomy into Unstructured, Off-Road Environments: Dataset Challenges
Source abstract
Abstract
As with the broad field of deep learning, autonomy is a research topic that has experienced a heavy explosion in attention from both the scientific and commercial industries due to its potential for the advancement of humanity in many cross-cutting disciplines. Recent advancements in computer vision-based autonomy has highlighted the potential for the realization of increasingly sophisticated autonomous ground vehicles for both commercial and non-traditional applications, such as grocery delivery. Part of the success of these technologies has been a boon in the abundance of training data that is available for training the autonomous behaviors associated with their autonomy software. These data abundance advantage is quickly diminished when an application moves from structured environments, i.e., well-defined city road networks, highways, street signage, etc., into unstructured environments, i.e., cross-country, off-road, non-traditional terrains. Herein, we aim to present insights, from a dataset perspective, into how the scientific community can begin to expand autonomy into unstructured environments, while highlighting some of the key challenges that are presented with such a dynamic and ever-changing environment. Finally, a foundation is laid for the creation of a robust off-road dataset being developed by the Engineer Research and Development Center and Mississippi State University’s Center for Advanced Vehicular Systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Tunneling and Rock Mechanics · Occupational Health and Safety Research · Robot Manipulation and Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
U.S. Army Engineer Research and Development Center, Vicksburg, MS 39180-6199, USA
Funding
U.S. Army Engineer Research and Development Center
References (41)
Show source-provided references
1. Gao, Y., Lin, T., Borrelli, F., Tseng, E., and Hrovat, D. (2010, January 12–15). Predictive control of autonomous ground vehicles with obstacle avoidance on slippery roads. Proceedings of the Dynamic Systems and Control Conference, Cambridge, MA, USA.10.1115/DSCC2010-4263 ↗
2. Febbo, H., Liu, J., Jayakumar, P., Stein, J.L., and Ersal, T. (2017, January 24–26). Moving obstacle avoidance for large, high-speed autonomous ground vehicles. Proceedings of the 2017 American Control Conference (ACC), Seattle, DC, USA.10.23919/ACC.2017.7963821 ↗
3. Guastella, D.C., and Muscato, G. (2020). Learning-based methods of perception and navigation for ground vehicles in unstructured environments: A review. Sensors, 21.10.3390/s21010073 ↗
4. Islam, F., Nabi, M., and Ball, J.E. (2022). Off-road detection analysis for autonomous ground vehicles: A review. Sensors, 22.10.3390/s22218463 ↗
5. Path planning and path tracking for collision avoidance of autonomous ground vehicles10.1109/JSYST.2021.3085479 ↗
6. Terapaptommakol, W., Phaoharuhansa, D., Koowattanasuchat, P., and Rajruangrabin, J. (2022). Design of obstacle avoidance for autonomous vehicle using deep Q-network and CARLA simulator. World Electr. Veh. J., 13.10.3390/wevj13120239 ↗
7. Wang, N., Li, X., Zhang, K., Wang, J., and Xie, D. (2024). A survey on path planning for autonomous ground vehicles in unstructured environments. Machines, 12.10.3390/machines12010031 ↗
8. (2024, August 01). Tesla Vehicle Safety Report. Available online: https://www.tesla.com/VehicleSafetyReport.
9. (2024, August 01). Waymo Significantly Outperforms Comparable Human Benchmarks over 7+ Million Miles of Rider-Only Driving. Available online: https://waymo.com/blog/2023/12/waymo-significantly-outperforms-comparable-human-benchmarks-over-7-million/.
10. A survey of deep learning techniques for autonomous driving10.1002/rob.21918 ↗
11. Ni, J., Chen, Y., Chen, Y., Zhu, J., Ali, D., and Cao, W. (2020). A survey on theories and applications for self-driving cars based on deep learning methods. Appl. Sci., 10.10.3390/app10082749 ↗
12. Comparative study of end-to-end deep learning methods for self-driving car
13. Deep learning for object detection and scene perception in self-driving cars: Survey, challenges, and open issues10.1016/j.array.2021.100057 ↗
14. Wijayathunga, L., Rassau, A., and Chai, D. (2023). Challenges and solutions for autonomous ground robot scene understanding and navigation in unstructured outdoor environments: A review. Appl. Sci., 13.10.20944/preprints202304.0373.v1 ↗
15. Wigness, M., Eum, S., Rogers, J.G., Han, D., and Kwon, H. (2019, January 3–8). A RUGD Dataset for Autonomous Navigation and Visual Perception in Unstructured Outdoor Environments. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Macau, China.10.1109/IROS40897.2019.8968283 ↗
16. Jiang, P., Osteen, P., Wigness, M., and Saripalli, S. (2020). RELLIS-3D Dataset: Data, Benchmarks and Analysis. arXiv.10.1109/ICRA48506.2021.9561251 ↗
17. Sivaprakasam, M., Maheshwari, P., Castro, M.G., Triest, S., Nye, M., Willits, S., Saba, A., Wang, W., and Scherer, S. (2024). TartanDrive 2.0: More Modalities and Better Infrastructure to Further Self-Supervised Learning Research in Off-Road Driving Tasks. arXiv.10.1109/ICRA57147.2024.10611265 ↗
18. Triest, S., Sivaprakasam, M., Wang, S.J., Wang, W., Johnson, A.M., and Scherer, S. (2022, January 23–27). Tartandrive: A large-scale dataset for learning off-road dynamics models. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.10.1109/ICRA46639.2022.9811648 ↗
19. Kishore, A., Choe, T.E., Kwon, J., Park, M., Hao, P., and Mittel, A. (2021, January 11–17). Synthetic data generation using imitation training. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCVW54120.2021.00342 ↗
20. Jaipuria, N., Zhang, X., Bhasin, R., Arafa, M., Chakravarty, P., Shrivastava, S., Manglani, S., and Murali, V.N. (2020, January 14–19). Deflating dataset bias using synthetic data augmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00394 ↗
21. Using synthetic data to enhance the accuracy of fingerprint-based localization: A deep learning approach10.1109/LSENS.2020.2971555 ↗
22. The vehicle testing based on digital twins theory for autonomous vehicles10.1109/JRFID.2022.3211565 ↗
23. Price, S.R., Price, S.R., and Anderson, D.T. (2019, January 23–26). Introducing fuzzy layers for deep learning. Proceedings of the 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), New Orleans, LA, USA.10.1109/FUZZ-IEEE.2019.8858790 ↗
24. A comprehensive review of deep neuro-fuzzy system architectures and their optimization methods10.1007/s00521-021-06807-9 ↗
25. Imitation learning for agile autonomous driving10.1177/0278364919880273 ↗
26. Chen, J., Yuan, B., and Tomizuka, M. (2019, January 3–8). Deep imitation learning for autonomous driving in generic urban scenarios with enhanced safety. Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.10.1109/IROS40897.2019.8968225 ↗
27. A survey on imitation learning techniques for end-to-end autonomous vehicles10.1109/TITS.2022.3144867 ↗
28. Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving10.1016/j.trc.2020.102662 ↗
29. Efficient deep reinforcement learning with imitative expert priors for autonomous driving10.1109/TNNLS.2022.3142822 ↗
30. Deep reinforcement learning for autonomous driving: A survey10.1109/TITS.2021.3054625 ↗
31. Hua, J., Zeng, L., Li, G., and Ju, Z. (2021). Learning for a robot: Deep reinforcement learning, imitation learning, transfer learning. Sensors, 21.10.3390/s21041278 ↗
32. Is imitation learning the route to humanoid robots?10.1016/S1364-6613(99)01327-3 ↗
33. Johns, E. (June, January 30). Coarse-to-fine imitation learning: Robot manipulation from a single demonstration. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, China.10.1109/ICRA48506.2021.9560942 ↗
34. Data-driven product design evaluation method based on multi-stage artificial neural network10.1016/j.asoc.2021.107117 ↗
35. Multi-stage optimized machine learning framework for network intrusion detection10.1109/TNSM.2020.3014929 ↗
36. Vemulapalli, R., Pouransari, H., Faghri, F., Mehta, S., Farajtabar, M., Rastegari, M., and Tuzel, O. (2024, January 21). Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models. Proceedings of the ICML, Vienna, Austria.
37. Ross, S., and Bagnell, J.A. (2014). Reinforcement and imitation learning via interactive no-regret learning. arXiv.
38. Reddy, S., Dragan, A.D., and Levine, S. (2019). Sqil: Imitation learning via reinforcement learning with sparse rewards. arXiv.
39. Zhu, Y., Wang, Z., Merel, J., Rusu, A., Erez, T., Cabi, S., Tunyasuvunakool, S., Kramár, J., Hadsell, R., and de Freitas, N. (2018). Reinforcement and imitation learning for diverse visuomotor skills. arXiv.10.15607/RSS.2018.XIV.009 ↗
40. Le, H., Jiang, N., Agarwal, A., Dudík, M., Yue, Y., and Daumé III, H. (2018, January 10–15). Hierarchical imitation and reinforcement learning. Proceedings of the International Conference on Machine Learning. PMLR, Stockholm, Sweden.
41. Sallab, A.E., Abdou, M., Perot, E., and Yogamani, S. (2017). Deep reinforcement learning framework for autonomous driving. arXiv.10.2352/ISSN.2470-1173.2017.19.AVM-023 ↗