Hourglass 3D CNN for Stereo Disparity Estimation for Mobile Robots
Source abstract
Abstract
Stereo cameras allow mobile robots to perceive depth in their surroundings by capturing two separate images from slightly different perspectives. This is necessary for tasks such as obstacle avoidance, navigation, and spatial mapping. By utilizing a convolutional neural network (CNN), existing works in stereo cameras based on depth estimation have achieved superior results. However, the critical requirement for depth estimation for mobile robots is to have an optimal tradeoff between computational cost and accuracy. To achieve such a tradeoff, attention-aware feature aggregation (AAFS) has been proposed for real-time stereo matching on edge devices. AAFS includes multistage feature extraction, an attention module, and a 3D CNN architecture. However, its 3D CNN architecture learns contextual information ineffectively. In this paper, a deep encoder–decoder architecture is applied to an AAFS 3D CNN to improve depth estimation accuracy. Through evaluation, it is proven that the proposed 3D CNN architecture provides significantly better accuracy while keeping the inference time comparable to that of AAFS.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Vision and Imaging · Image Processing Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechanical Engineering, Hanoi University of Science and Technology, Hanoi 100000, Vietnam
Graduate School of Engineering and Science, Shibaura Institute of Technology, Tokyo 135-8548, Japan
Department of Information and Communications Engineering, Shibaura Institute of Technology, Tokyo 135-8548, Japan
References (29)
Show source-provided references
1. Campbell, S., O’Mahony, N., Carvalho, A., Krpalkova, L., Riordan, D., and Walsh, J. (2020, January 12–15). Path Planning Techniques for Mobile Robots A Review. Proceedings of the 2020 6th International Conference on Mechatronics and Robotics Engineering (ICMRE), Barcelona, Spain.10.1109/ICMRE49073.2020.9065187 ↗
2. Vasic, M., and Billard, A. (2013, January 6–10). Safety Issues in Human-robot Interactions. Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany.10.1109/ICRA.2013.6630576 ↗
3. Wei, B., Gao, J., Li, K., Fan, Y., Gao, X., and Gao, B. (2009, January 22–25). Indoor Mobile Robot Obstacle Detection Based on Linear Structured Light Vision System. Proceedings of the 2008 IEEE International Conference on Robotics and Biomimetics, Bangkok, Thailand.
4. An information-based exploration strategy for environment mapping with mobile robots10.1016/j.robot.2009.11.005 ↗
5. Ekvall, S., Jensfelt, P., and Kragic, D. (2006, January 9–15). Integrating Active Mobile Robot Object Recognition and Slam in Natural Environments. Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China.10.1109/IROS.2006.282389 ↗
6. Using Real-time Stereo Vision for Mobile Robot Navigation10.1023/A:1008987612352 ↗
7. Scharstein, D., Szeliski, R., and Zabih, R. (2001, January 9–10). A Taxonomy and Evaluation of Dense Two-frame Stereo Correspondence Algorithms. Proceedings of the IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001), Kauai, Hawaii.
8. Flynn, J., Neulander, I., Philbin, J., and Snavely, N. (2016, January 27–30). Deep Stereo: Learning to Predict New Views from the World’s Imagery. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.595 ↗
9. Kendall, A., Martirosyan, H., Dasgupta, S., Henry, P., Kennedy, R., Bachrach, A., and Bry, A. (2017, January 22–29). End-to-End Learning of Geometry and Context for Deep Stereo Regression. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.17 ↗
10. Chang, J.R., and Chen, Y.S. (2018, January 18–22). Pyramid Stereo Matching Network. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00567 ↗
11. Zhang, F., Prisacariu, V., Yang, R., and Torr, P.H. (2019, January 15–20). GA-Net: Guided Aggregation Net for End-to-end Stereo Matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00027 ↗
12. Chang, J.R., Chang, P.C., and Chen, Y.S. Attention-Aware Feature Aggregation for Real-Time Stereo Matching on Edge Devices. Proceedings of the Computer Vision–ACCV 2020.10.1007/978-3-030-69525-5_22 ↗
13. Khamis, S., Fanello, S., Rhemann, C., Kowdle, A., Valentin, J., and Izadi, S. (2018, January 8–14). StereoNet: Guided Hierarchical Refinement for Real-Time Edge-Aware Depth Prediction. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01267-0_35 ↗
14. Poppinga, B., and Laue, T. JET-Net: Real-time Object Detection for Mobile Robots. Proceedings of the RoboCup 2019: Robot World Cup XXIII 23.10.1007/978-3-030-35699-6_18 ↗
15. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation10.1109/TPAMI.2016.2644615 ↗
16. Newell, A., Yang, K., and Deng, J. (2016, January 11–14). Stacked Hourglass Networks for Human Pose Estimation. Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands. Proceedings, Part VIII 14.10.1007/978-3-319-46484-8_29 ↗
17. Stereo matching by training a convolutional neural network to compare image patches
18. Guo, X., Yang, K., Yang, W., Wang, X., and Li, H. (2019, January 15–20). Group-wise Correlation Stereo Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00339 ↗
19. Xu, H., and Zhang, J. (2020, January 14–19). AANet: Adaptive Aggregation Network for Efficient Stereo Matching. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00203 ↗
20. Duggal, S., Wang, S., Ma, W., Hu, R., and Urtasun, R. (November, January 27). DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00448 ↗
21. Gu, X., Fan, Z., Zhu, S., Dai, Z., Tan, F., and Tan, P. (2020, January 14–19). Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00257 ↗
22. Haase, D., and Amthor, M. (2020, January 13–19). Rethinking Depthwise Separable Convolutions: How Intra-Kernel Correlations Lead to Improved MobileNets. Proceedings of the The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01461 ↗
23. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
24. Mayer, N., Ilg, E., Häusser, P., Fischer, P., Cremers, D., Dosovitskiy, A., and Brox, T. (2016, January 27–30). A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.438 ↗
25. Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16–21). Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA.10.1109/CVPR.2012.6248074 ↗
26. Menze, M., and Geiger, A. (2015, January 7–12). Object Scene Flow for Autonomous Vehicles. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298925 ↗
27. Pytorch: An imperative style, high-performance deep learning library
28. Haeusler, R., Nair, R., and Kondermann, D. (2013, January 23–28). Ensemble Learning for Confidence Measures in Stereo Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA.10.1109/CVPR.2013.46 ↗
29. Xu, G., Wang, Y., Cheng, J., Tang, J., and Yang, X. (2022). Accurate and Efficient Stereo Matching via Attention Concatenation Volume. arXiv.