Training Data for Stereo Matching Algorithms Based on Neural Networks and a Method for Data Evaluation
Source abstract
Abstract
This paper addresses the problem of selecting data for learning stereo matching algorithms. The paper presents an overview of currently available learning datasets, including synthetic data and data from real environments. Stereo matching algorithms based on neural networks require high quality learning data in order to provide expected results, which have a form of disparity maps. There are hundreds of stereo matching algorithms for processing images from stereo cameras. However, a significant problem with this 3D data processing technology is that there is a relatively low availability of learning data with dense ground truth from real environments. The paper also introduces an evaluation method for estimating the quality of input data. The method considers features such as the quality of calibration, the density of ground truth, and the occurrence of occluded areas. The research presented in this paper is aimed at developing stereo matching methods applicable not only to benchmarks of this kind of algorithms but also to applications in real environments.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Vision and Imaging · Advanced Image and Video Retrieval Techniques · Satellite Image Processing and Photogrammetry
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology, 80-233 Gdansk, Poland
Funding
Gdańsk University of Technology
References (67)
Show source-provided references
1. Guo, X., Yang, K., Yang, W., Wang, X., and Li, H. (2019, January 15–20). Group-wise Correlation Stereo Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00339 ↗
2. Knöbelreiter, P., Sormann, C., Shekhovtsov, A., Fraundorfer, F., and Pock, T. (2020, January 14–19). Belief Propagation Reloaded: Learning BP-Layers for Labeling Problems. Proceedings of the The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00792 ↗
3. Stereo matching by training a convolutional neural network to compare image patches
4. A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms10.1023/A:1014573219977 ↗
5. Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16–21). Are we ready for autonomous driving? The KITTI vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.10.1109/CVPR.2012.6248074 ↗
6. Kaczmarek, A.L. (2025). A Method for Adapting Stereo Matching Algorithms to Real Environments. Appl. Sci., 15.10.3390/app15074070 ↗
7. Kaczmarek, A.L., and Blaschitz, B. (2021). Equal Baseline Camera Array—Calibration, Testbed and Applications. Appl. Sci., 11.10.3390/app11188464 ↗
8. Błaszczak-Bąk, W., Kamiński, W., Bednarczyk, M., Suchocki, C., and Masiero, A. (2025). Real-Time DTM Generation with Sequential Estimation and OptD Method. Appl. Sci., 15.10.3390/app15074068 ↗
9. Zhang, Z., Wei, C., Wu, G., and Barth, M.J. (2025). Vulnerable Road User Detection for Roadside-Assisted Safety Protection: A Comprehensive Survey. Appl. Sci., 15.10.3390/app15073797 ↗
10. Gupta, M., Yin, Q., and Nayar, S.K. (2013, January 1–8). Structured Light in Sunlight. Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.73 ↗
11. Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. (2006, January 17–22). A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), New York, NY, USA.
12. (2025, October 17). Middlebury Stereo Evaluation-Version 3. Available online: https://vision.middlebury.edu/stereo/eval3/.
13. (2025, October 17). KITTI Vision Benchmark Suite. Available online: https://www.cvlibs.net/datasets/kitti/.
14. 3D Vision System for a Robotic Arm Based on Equal Baseline Camera Array10.1007/s10846-019-01117-8 ↗
15. Bradski, D.G.R., and Kaehler, A. (2008). Learning Opencv, O’Reilly Media, Inc.. [1st ed.].
16. Jiang, X., Hornegger, J., and Koch, R. (2014, January 2–5). High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth. Proceedings of the Pattern Recognition, GCPR 2014, Münster, Germany.
17. Vision meets Robotics: The KITTI Dataset10.1177/0278364913491297 ↗
18. Schöps, T., Schönberger, J.L., Galliani, S., Sattler, T., Schindler, K., Pollefeys, M., and Geiger, A. (2017, January 21–26). A Multi-View Stereo Benchmark with High-Resolution Images and Multi-Camera Videos. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.272 ↗
19. Zama Ramirez, P., Tosi, F., Poggi, M., Salti, S., Di Stefano, L., and Mattoccia, S. (2022., January 18–24). Open Challenges in Deep Stereo: The Booster Dataset. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.02049 ↗
20. Booster: A Benchmark for Depth from Images of Specular and Transparent Surfaces10.1109/TPAMI.2023.3323858 ↗
21. (2025, October 17). NTIRE 2025: HR Depth from Images of Specular and Transparent Surfaces. Available online: https://cvlab-unibo.github.io/booster-web/ntire25.html.
22. Wang, Q., Wu, D., Liu, W., Lou, M., Jiang, H., Ying, Y., and Zhou, M. (2023). PlantStereo: A High Quality Stereo Matching Dataset for Plant Reconstruction. Agriculture, 13.10.3390/agriculture13020330 ↗
23. InStereo2K: A large real dataset for stereo matching in indoor scenes10.1007/s11432-019-2803-x ↗
24. Hua, Y., Kohli, P., Uplavikar, P., Ravi, A., Gunaseelan, S., Orozco, J., and Li, E. (2020). Holopix50k: A Large-Scale In-the-wild Stereo Image Dataset. arXiv.
25. Wang, Y., Wang, L., Yang, J., An, W., and Guo, Y. (2019, January 27–28). Flickr1024: A Large-Scale Dataset for Stereo Image Super-Resolution. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea.10.1109/ICCVW.2019.00478 ↗
26. Wang, C., Lucey, S., Perazzi, F., and Wang, O. (2019, January 16–19). Web Stereo Video Supervision for Depth Prediction from Dynamic Scenes. Proceedings of the 2019 International Conference on 3D Vision (3DV), Québec City, QC, Canada.10.1109/3DV.2019.00046 ↗
27. Shin, U., Park, J., and Kweon, I.S. (2023, January 18–22). Deep Depth Estimation From Thermal Image. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00107 ↗
28. The Multivehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception10.1109/LRA.2018.2800793 ↗
29. Yang, G., Song, X., Huang, C., Deng, Z., Shi, J., and Zhou, B. (2019, January 15–20). DrivingStereo: A Large-Scale Dataset for Stereo Matching in Autonomous Driving Scenarios. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00099 ↗
30. Chang, M.F., Lambert, J.W., Sangkloy, P., Singh, J., Bak, S., Hartnett, A., Wang, D., Carr, P., Lucey, S., and Ramanan, D. (2019, January 15–20). Argoverse: 3D Tracking and Forecasting with Rich Maps. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00895 ↗
31. Wilson, B., Qi, W., Agarwal, T., Lambert, J., Singh, J., Khandelwal, S., Pan, B., Kumar, R., Hartnett, A., and Pontes, J.K. (2021, January 5–14). Argoverse 2: Next Generation Datasets for Self-driving Perception and Forecasting. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS Datasets and Benchmarks 2021), Virtual Conference.
32. Lambert, J., and Hays, J. (2021, January 5–14). Trust, but Verify: Cross-Modality Fusion for HD Map Change Detection. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS Datasets and Benchmarks 2021), Virtual Conference.
33. Sivaprakasam, M., Maheshwari, P., Castro, M.G., Triest, S., Nye, M., Willits, S., Saba, A., Wang, W., and Scherer, S. (2024). TartanDrive 2.0: More Modalities and Better Infrastructure to Further Self-Supervised Learning Research in Off-Road Driving Tasks. arXiv.10.1109/ICRA57147.2024.10611265 ↗
34. 1 Year, 1000km: The Oxford RobotCar Dataset10.1177/0278364916679498 ↗
35. Maddern, W., Pascoe, G., Gadd, M., Barnes, D., Yeomans, B., and Newman, P. (2020). Real-time Kinematic Ground Truth for the Oxford RobotCar Dataset. arXiv.
36. The EuRoC micro aerial vehicle datasets10.1177/0278364915620033 ↗
37. A Comprehensive Implementation of Road Surface Classification for Vehicle Driving Assistance: Dataset, Models, and Deployment10.1109/TITS.2023.3264588 ↗
38. High-recall calibration monitoring for stereo cameras10.1007/s10044-024-01264-1 ↗
39. Wen, B., Trepte, M., Aribido, J., Kautz, J., Gallo, O., and Birchfield, S. (2025). FoundationStereo: Zero-Shot Stereo Matching. arXiv.10.1109/CVPR52734.2025.00495 ↗
40. Tremblay, J., To, T., and Birchfield, S. (2018, January 18–22). Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Los Alamitos, CA, USA.10.1109/CVPRW.2018.00275 ↗
41. Mayer, N., Ilg, E., Häusser, P., Fischer, P., Cremers, D., Dosovitskiy, A., and Brox, T. (2016, January 27–30). A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.438 ↗
42. Mehl, L., Schmalfuss, J., Jahedi, A., Nalivayko, Y., and Bruhn, A. (2023, January 18–22). Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00482 ↗
43. Schmalfuss, J., Oei, V., Mehl, L., Bartsch, M., Agnihotri, S., Keuper, M., and Bruhn, A. (2025). RobustSpring: Benchmarking Robustness to Image Corruptions for Optical Flow, Scene Flow and Stereo. arXiv.
44. Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., and Schmid, C. (2012, January 7–13). A Naturalistic Open Source Movie for Optical Flow Evaluation. Proceedings of the Computer Vision–ECCV 2012, Florence, Italy.10.1007/978-3-642-33712-3 ↗
45. Cabon, Y., Murray, N., and Humenberger, M. (2020). Virtual KITTI 2. arXiv.
46. Gaidon, A., Wang, Q., Cabon, Y., and Vig, E. (2016, January 27–30). Virtual worlds as proxy for multi-object tracking analysis. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.470 ↗
47. Deschaud, J. (2021). KITTI-CARLA: A KITTI-like dataset generated by CARLA Simulator. arXiv.
48. Dosovitskiy, A., Ros, G., Codevilla, F., López, A.M., and Koltun, V. (2017). CARLA: An Open Urban Driving Simulator. arXiv.
49. Wang, Q., Zheng, S., Yan, Q., Deng, F., Zhao, K., and Chu, X. (2021, January 5–9). IRS: A Large Naturalistic Indoor Robotics Stereo Dataset to Train Deep Models for Disparity and Surface Normal Estimation. Proceedings of the 2021 IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, China.10.1109/ICME51207.2021.9428423 ↗
50. Karaev, N., Rocco, I., Graham, B., Neverova, N., Vedaldi, A., and Rupprecht, C. (2023). DynamicStereo: Consistent Dynamic Depth from Stereo Videos. arXiv.10.1109/CVPR52729.2023.01271 ↗
51. Straub, J., Whelan, T., Ma, L., Chen, Y., Wijmans, E., Green, S., Engel, J.J., Mur-Artal, R., Ren, C.Y., and Verma, S. (2019). The Replica Dataset: A Digital Replica of Indoor Spaces. arXiv.
52. Fisher, R. (2025, October 17). CVonline: Image Databases. Available online: http://homepages.inf.ed.ac.uk/rbf/CVonline/Imagedbase.htm.
53. Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., and Schmid, C. (2012, January 7–13). Indoor Segmentation and Support Inference from RGBD Images. Proceedings of the Computer Vision–ECCV 2012, Florence, Italy.10.1007/978-3-642-33709-3 ↗
54. The ApolloScape Open Dataset for Autonomous Driving and Its Application10.1109/TPAMI.2019.2926463 ↗
55. Wang, S., Bai, M., Máttyus, G., Chu, H., Luo, W., Yang, B., Liang, J., Cheverie, J., Fidler, S., and Urtasun, R. (2016). TorontoCity: Seeing the World with a Million Eyes. arXiv.10.1109/ICCV.2017.327 ↗
56. Kondermann, D., Nair, R., Honauer, K., Krispin, K., Andrulis, J., Brock, A., Güssefeld, B., Rahimimoghaddam, M., Hofmann, S., and Brenner, C. (July, January 26). The HCI Benchmark Suite: Stereo and Flow Ground Truth with Uncertainties for Urban Autonomous Driving. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Las Vegas, NV, USA.10.1109/CVPRW.2016.10 ↗
57. Li, J., Wang, P., Xiong, P., Cai, T., Yan, Z., Yang, L., Liu, J., Fan, H., and Liu, S. (2022, January 18–24). Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01578 ↗
58. Yang, G., Manela, J., Happold, M., and Ramanan, D. (2019, January 15–20). Hierarchical Deep Stereo Matching on High-Resolution Images. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00566 ↗
59. (2025, October 17). Camera Calibration and 3D Reconstruction-OpenCV 4.13.0 Documentation. Available online: https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html.
60. A flexible new technique for camera calibration10.1109/34.888718 ↗
61. Speeded-Up Robust Features (SURF)10.1016/j.cviu.2007.09.014 ↗
62. Liu, R., Zhang, H., Liu, M., Xia, X., and Hu, T. (2009, January 11–12). Stereo Cameras Self-Calibration Based on SIFT. Proceedings of the 2009 International Conference on Measuring Technology and Mechatronics Automation, Zhangjiajie, China.10.1109/ICMTMA.2009.338 ↗
63. Leutenegger, S., Chli, M., and Siegwart, R.Y. (2011, January 6–13). BRISK: Binary Robust invariant scalable keypoints. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.10.1109/ICCV.2011.6126542 ↗
64. Daniilidis, K., Maragos, P., and Paragios, N. (2010, January 5–11). BRIEF: Binary Robust Independent Elementary Features. Proceedings of the Computer Vision–ECCV 2010, Crete, Greece.10.1007/978-3-642-15555-0 ↗
65. Alcantarilla, P.F., Nuevo, J., and Bartoli, A. (2013, January 9–13). Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces. Proceedings of the British Machine Vision Conference, Bristol, UK.10.5244/C.27.13 ↗
66. Karami, E., Prasad, S., and Shehata, M. (2017). Image Matching Using SIFT, SURF, BRIEF and ORB: Performance Comparison for Distorted Images. arXiv.
67. (2025, November 23). OpenCV 4.13.0-dev: Basics of Brute-Force Matcher. Available online: https://docs.opencv.org/4.x/dc/dc3/tutorial_py_matcher.html.