Comparative Analysis of Correction Methods for Multi-Camera 3D Image Processing System and Its Application Design in Safety Improvement on Hot-Working Production Line
Source abstract
Abstract
The paper presents the results of research focused on configuring a system for stereoscopic view capturing and processing. The system is being developed for use in staff training scenarios based on Virtual Reality (VR), where high-quality, distortion-free imagery is essential. This research addresses key challenges in image distortion, including the fish-eye effect and other aberrations. In addition, it considers the computational and bandwidth efficiency required for effective and economical streaming and real-time display of recorded content. Measurements and calculations were performed using a selected set of cameras, adapters, and lenses, chosen based on predefined criteria. A comparative analysis was conducted between the nearest-neighbour linear interpolation method and a third-order polynomial interpolation (ABCD polynomial). These methods were tested and evaluated using three different computational approaches, each aimed at optimizing data processing efficiency critical for real-time image correction. Images captured during real-time video transmission—processed using the developed correction techniques—are presented. In the final sections, the paper describes the configuration of an innovative VR-based training system incorporating an edge computing device. A case study involving a factory producing wheel rims is also presented to demonstrate the practical application of the system.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Advanced Manufacturing Technologies, Faculty of Mechanical Engineering, Wroclaw University of Science and Technology, 27 Wybrzeże Stanisława Wyspiańskiego, 50-370 Wrocław, Poland
Funding
European Union
References (64)
Show source-provided references
1. SpinVR: Towards Live-Streaming 3D Virtual Reality Video10.1145/3130800.3130836 ↗
2. Luhmann, T., Robson, S., Kyle, S., Boehm, J., and de Gruyter, W. (2020). Close-Range Photogrammetry and 3D Imaging, Gruyter GmbH & Co KG.10.1515/9783110607253 ↗
3. Temporal Lensing: An Interactive and Scalable Technique for Web3D/WebXR Applications in Cultural Heritage
4. An 18 megapixel 4.3" 1443 ppi 120 Hz OLED display for wide field of view high acuity head mounted displays10.1002/jsid.658 ↗
5. Gao, Z., Hwang, A., Zhai, G., and Peli, E. (2018). Correcting geometric distortions in stereoscopic 3D imaging. PLoS ONE, 13.10.1371/journal.pone.0205032 ↗
6. Dansereau, D.G., Schuster, G., Ford, J., and Wetzstein, G. (2017, January 21–26). A Wide-Field-of-View Monocentric Light Field Camera. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.400 ↗
7. Limits of peripheral acuity and implications for VR system design10.1002/jsid.730 ↗
8. Temporal Requirements for VR Displays to Create a More Comfortable and Immersive Visual Experience
9. Jump: Virtual Reality Video
10. Evaluation of a low-cost approach to 2-D digital image correlation vs. a commercial stereo-DIC system in Brazilian testing of soil specimens
11. Hughes, C., Jones, E., Glavin, M., and Denny, P. (2009). Validation of Polynomial-based Equidistance Fish-Eye Models. IET Irish Signals and Systems Conference (ISSC), IET.10.1049/cp.2009.1700 ↗
12. Fisheye lens designs and their relative performance10.1117/12.405226 ↗
13. A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses10.1109/TPAMI.2006.153 ↗
14. Review of geometric distortion compensation in automotive camera systems
15. Scaramuzza, D., Martinelli, A., and Siegwart, R. (2006, January 4–7). A flexible technique for accurate omnidirectional camera calibration and structure from motion. Proceedings of the IEEE International Conference on Computer Vision Systems, New York, NY, USA.10.1109/ICVS.2006.3 ↗
16. Rong, Y., Su, S., and Yan, S. (2016, January 25–28). Radial distortion correction with deep convolutional neural networks. Proceedings of the IEEE International Conference on Image Processing, Phoenix, AZ, USA.
17. New calibration methods and datasets for omnidirectional cameras
18. Madejski, G., Zbytniewski, S., Kurowski, M., Gradolewski, D., Kaoka, W., and Kulesza, W.J. (2024). Lens Distortion Measurement and Correction for Stereovision Multi-Camera System. Eng. Proc., 82.10.3390/ecsa-11-20457 ↗
19. Hughes, C., Glavin, M., Jones, E., and Denny, P. (2008, January 18–19). Review of Geometric Distortion Compensation in Fish-Eye Cameras. Proceedings of the IET Irish Signals and Systems Conference (ISSC), Galway, Ireland.10.1049/cp:20080656 ↗
20. Slama, C. (1980). Manual of Photogrammetry. American Society of Photogrammetry, American Society of Photogrammetry. [4th ed.].
21. A versatile camera calibration technique for high accuracy 3d machine vision metrology using off-the-shelf TV cameras and lenses10.1109/JRA.1987.1087109 ↗
22. Mallon, J., and Whelan, P.F. (2004, January 26–26). Precise radial un-distortion of images. Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK.10.1109/ICPR.2004.1333995 ↗
23. Nonmetric calibration of camera lens distortion: Differential methods and robust estimation10.1109/TIP.2005.846025 ↗
24. Fitzgibbon, A.W. (2001, January 8–14). Simultaneous linear estimation of multiple view geometry and lens distortion. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA.
25. Design of an efficient VLSI architecture for non-linear spatial warping of wide-angle camera images10.1016/j.sysarc.2004.05.001 ↗
26. Shah, S., and Aggarwal, J.K. (1994, January 8–13). A simple calibration procedure for fisheye (high distortion) lens camera. Proceedings of the 1994 IEEE International Conference on Robotics and Automation, San Diego, CA, USA.
27. Alternative models for fish-eye lenses10.1016/0167-8655(94)00115-J ↗
28. Straight lines have to be straight: Automatic calibration and removal of distortion from scenes of structured environments10.1007/PL00013269 ↗
29. Ishii, C., Sudo, Y., and Hashimoto, H. (2003, January 20–24). An image conversion algorithm from fisheye image to perspective image for human eyes. Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Kobe, Japan.
30. Chen, L., Li, S., Bai, Q., Yang, J., Jiang, S., and Miao, Y. (2021). Review of Image Classification Algorithms Based on Convolutional Neural Networks. Remote Sens., 13.10.3390/rs13224712 ↗
31. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery10.3390/rs71114680 ↗
32. Hydra: An ensemble of convolutional neural networks for geospatial land classification10.1109/TGRS.2019.2906883 ↗
33. Scene classification with recurrent attention of VHR remote sensing images10.1109/TGRS.2018.2864987 ↗
34. Chen, G., Zhang, X., Tan, X., Cheng, Y., Dai, F., Zhu, K., Gong, Y., and Wang, Q. (2018). Training small networks for scene classification of remote sensing images via knowledge distillation. Remote Sens., 10.10.3390/rs10050719 ↗
35. Mangasarian, O.L., and Musicant, D.R. (2001). Data Discrimination via Nonlinear Generalized Support Vector Machines. Complementarity: Applications, Algorithms and Extensions, Springer.10.1007/978-1-4757-3279-5_11 ↗
36. ImageNet Classification with Deep Convolutional Neural Networks
37. Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
38. Representation Learning: A Review and New Perspectives10.1109/TPAMI.2013.50 ↗
39. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 15). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
40. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). Identity Mappings in Deep Residual Networks. Computer Vision—ECCV 2016, Springer International Publishing.10.1007/978-3-319-46466-4 ↗
41. Zagoruyko, S., and Komodakis, N. (2016). Wide Residual Networks. arXiv.10.5244/C.30.87 ↗
42. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. (2017, January 4–9). Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.11231 ↗
43. Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K. (2017, January 21–26). Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
44. Fully Convolutional Networks for Semantic Segmentation
45. Hu, J., Shen, L., Sun, G., and Albanie, S. (2019, January 15–20). Squeeze-and-Excitation Networks. Proceedings of the IEEE Transactions on Pattern Analysis and Machine Intelligence, Long Beach, CA, USA.
46. Park, J., Woo, S., Lee, J.Y., and Kweon, I.S. (2018). BAM: Bottleneck Attention Module. arXiv.
47. Hu, J., Shen, L., Albanie, S., Sun, G., and Vedaldi, A. (2018). Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks, NIPS’18; Curran Associates Inc.
48. Li, X., Wang, W., Hu, X., and Yang, J. (2019, January 15–20). Selective Kernel Networks. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00060 ↗
49. Ionescu, C., Vantzos, O., and Sminchisescu, C. (2015, January 7–13). Matrix Backpropagation for Deep Networks with Structured Layers. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.339 ↗
50. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., and Hu, Q. (2019). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. arXiv.10.1109/CVPR42600.2020.01155 ↗
51. Hou, Q., Zhou, D., and Feng, J. (2021). Coordinate Attention for Efficient Mobile Network Design. arXiv.10.1109/CVPR46437.2021.01350 ↗
52. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv.
53. Howard, A., Pang, R., Adam, H., Le, Q., Sandler, M., Chen, B., Wang, W., Chen, L.C., Tan, M., and Chu, G. (November, January 27). Searching for MobileNetV3. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
54. Chollet, F. (2017, January 21–26). Xception: Deep Learning with Depthwise Separable Convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
55. Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., and Le, Q.V. (2018). MnasNet: Platform-Aware Neural Architecture Search for Mobile. arXiv.10.1109/CVPR.2019.00293 ↗
56. Automatic Panoramic Image Stitching Using Invariant Features10.1007/s11263-006-0002-3 ↗
57. Low-cost 360 Stereo Photography and Video Capture10.1145/3072959.3073645 ↗
58. Richardt, C., Pritch, Y., Zimmer, H., and Sorkine-Hornung, A. (2013, January 23–28). Megastereo: Constructing High-Resolution Stereo Panoramas. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA.10.1109/CVPR.2013.166 ↗
59. Bourke, P. (2010, January 20–23). Capturing omni-directional stereoscopic spherical projections with a single camera. Proceedings of the 2010 16th International Conference on Virtual Systems and Multimedia, Seoul, Republic of Korea.10.1109/VSMM.2010.5665988 ↗
60. Rendering for an Interactive 360 Light Field Display10.1145/1276377.1276427 ↗
61. Omnistereo: Panoramic stereo imaging10.1109/34.910880 ↗
62. Tanaka, K., Hayashi, J., Inami, M., and Tachi, S. (2004, January 27–31). TWISTER: An immersive autostereoscopic display. Proceedings of the IEEE Virtual Reality 2004, Chicago, IL, USA.
63. Justyna, T., Olaf, C., José, M.M., and Pavlenko, I. (2019). Edge computing technologies as a crucial factor of successful Industry 4.0 growth. The case of live video data streaming. Advances in Manufacturing II Volume 1, Springer International Publishing. Solutions for Industry 4.0. Lecture Notes in Mechanical Engineering.
64. A review of distortion correction in wide-angle vision systems: Challenges and opportunities