Rectification for Stitched Images with Deformable Meshes and Residual Networks
Source abstract
Abstract
Image stitching is an important method for digital image processing, which is often prone to the problem of the irregularity of stitched images after stitching. And the traditional image cropping or complementation methods usually lead to a large number of information loss. Therefore, this paper proposes an image rectification method based on deformable mesh and residual network. The method aims to minimize the information loss at the edges of the spliced image and the information loss inside the image. Specifically, the method can select the most suitable mesh shape for residual network regression according to different images. Its loss function includes global loss and local loss, aiming to minimize the loss of image information within the grid and global target. The method in this paper not only greatly reduces the information loss caused by irregular shapes after image stitching, but also adapts to different images with various rigid structures. Meanwhile, its validation on the DIR-D dataset shows that the method outperforms the state-of-the-art methods in image rectification.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Digital Image Processing Techniques · Optical measurement and interference techniques · Industrial Vision Systems and Defect Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute of Remote Sensing and Geographic Information Systems, Peking University, No. 5 Summer Palace Road, Beijing 100871, China
Funding
National Key R&D Program for the 14th Five-Year Plan (Prevention and Control of Major Natural Disasters and Public Security, Shanjun Mao et al.)
References (64)
Show source-provided references
1. Bai, Z.W., Li, Y., Chen, X.H., Yi, T.T., Wei, W., Wozniak, M., and Damasevicius, R. (2020). Real-Time Video Stitching for Mine Surveillance Using a Hybrid Image Registration Method. Electronics, 9.10.3390/electronics9091336 ↗
2. Efficient and Accurate Stitching for 360° Dual-Fisheye Images and Videos10.1109/TIP.2021.3130531 ↗
3. TCPMFNet: An infrared and visible image fusion network with composite auto encoder and transformer-convolutional parallel mixed fusion strategy10.1016/j.infrared.2022.104405 ↗
4. Gao, H., Huang, Z.Q., Yang, H.P., Zhang, X.B., and Cen, C. (2023). Research on Improved Multi-Channel Image Stitching Technology Based on Fast Algorithms. Electronics, 12.10.3390/electronics12071700 ↗
5. Infrared-visible image fusion method based on sparse and prior joint saliency detection and LatLRR-FPDE10.1016/j.dsp.2023.103910 ↗
6. Subjective and Objective Quality Assessment of Stitched Images for Virtual Reality10.1109/TIP.2019.2921858 ↗
7. Wang, L., Yu, W., and Li, B. (2020, January 12–14). Multi-scenes Image Stitching Based on Autonomous Driving. Proceedings of the 4th IEEE Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Chongqing, China.10.1109/ITNEC48623.2020.9084886 ↗
8. Kinzig, C., Cortés, I., Fernández, C., and Lauer, M. (2022, January 4–7). Real-time Seamless Image Stitching in Autonomous Driving. Proceedings of the 25th International Conference of Information Fusion (FUSION), Linkoping, Sweden.10.23919/FUSION49751.2022.9841307 ↗
9. Single-Perspective Warps in Natural Image Stitching10.1109/TIP.2019.2934344 ↗
10. Flotation froth image recognition with convolutional neural networks10.1016/j.mineng.2018.12.011 ↗
11. Shi, J.Y., Dang, J., and Zuo, R.Z. (2021, January 11–18). Bridge damage cropping-and-stitching segmentation using fully convolutional network based on images from UAVs. Proceedings of the 10th International Conference on Bridge Maintenance, Safety and Management (IABMAS), Sapporo, Japan.10.1201/9780429279119-32 ↗
12. Reinforcement learning cropping method based on comprehensive feature and aesthetics assessment10.1049/ipr2.12420 ↗
13. Robust UAV Thermal Infrared Remote Sensing Images Stitching via Overlap-Prior-Based Global Similarity Prior Model10.1109/JSTARS.2020.3032011 ↗
14. Underwater laser positioning of targets outside the field of view based on a binocular vision10.1364/AO.499393 ↗
15. A regularised deep matrix factorised model of matrix completion for image restoration10.1049/ipr2.12553 ↗
16. Structure-Texture Consistent Painting Completion for Artworks10.1109/ACCESS.2023.3252892 ↗
17. Generative image completion with image-to-image translation10.1007/s00521-019-04253-2 ↗
18. Predictors of Completion of Clinically Necessary Radiologist Recommended Follow-Up Imaging: Assessment Using an Automated Closed- Loop Communication and Tracking Tool10.2214/AJR.22.28378 ↗
19. Xu, J.W., Li, F., Shao, C.C., and Li, X.L. (2023). Face Completion Based on Symmetry Awareness with Conditional GAN. Symmetry, 15.10.3390/sym15030663 ↗
20. Wang, H.T., Guo, E.T., Chen, F., and Chen, P.P. (2023). Depth Completion in Autonomous Driving: Adaptive Spatial Feature Fusion and Semi-Quantitative Visualization. Appl. Sci., 13.10.3390/app13179804 ↗
21. Nie, L., Lin, C.Y., Liao, K., Liu, S.C., and Zhao, Y. (2022, January 18–24). Deep Rectangling for Image Stitching: A Learning Baseline. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00565 ↗
22. ISGTA: An effective approach for multi-image stitching based on gradual transformation matrix10.1007/s11760-023-02609-9 ↗
23. Shape-Optimizing and Illumination-Smoothing Image Stitching10.1109/TMM.2018.2864576 ↗
24. A hybrid method of detecting flame from video stream10.1049/ipr2.12529 ↗
25. Image stitching using double features-based global similarity constraint and improved seam-cutting10.1007/s11042-023-15976-9 ↗
26. Rectangling Panoramic Images via Warping
27. Li, D.P., He, K.M., Sun, J., and Zhou, K. (2015, January 7–12). Geodesic-Preserving Method for Image Warping. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.
28. Modified Seam Carving by Changing Resizing Depending on the Object Size in Time and Space Domains
29. Patel, D., Shanmuganathan, S., and Raman, S. (2019, January 20–23). Adaptive Multiple-pixel Wide Seam Carving. Proceedings of the 25th National Conference on Communications (NCC), Bangalore, India.10.1109/NCC.2019.8732245 ↗
30. Hopfield network-based approach to detect seam-carved images and identify tampered regions10.1007/s00521-018-3463-8 ↗
31. Deep Convolutional Neural Network for Identifying Seam-Carving Forgery10.1109/TCSVT.2020.3037662 ↗
32. Robust detection of seam carving with low ratio via pixel adjacency subtraction and CNN-based transfer learning
33. Huang, Z., Lin, J., Yang, H., Wang, H., Bai, T., Liu, Q., and Pang, Y. (2020). An Algorithm Based on Text Position Correction and Encoder-Decoder Network for Text Recognition in the Scene Image of Visual Sensors. Sensors, 20.10.3390/s20102942 ↗
34. Zheng, Y., and Li, S. (2021, January 5–8). Two-stage Parallax Correction and Multi-stage Cross-view Fusion Network Based Stereo Image Super-Resolution. Proceedings of the IEEE International Conference on Visual Communications and Image Processing (VCIP)-Visual Communications in the Era of AI and Limited Resources, Munich, Germany.10.1109/VCIP53242.2021.9675418 ↗
35. Lee, Y., Lee, J., Ahn, H., and Jeon, M. (November, January 27). SNIDER: Single Noisy Image Denoising and Rectification for Improving License Plate Recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.
36. A decentralised approach to scene completion using distributed feature hashgram10.1007/s11042-019-08403-5 ↗
37. Global Convolutional Neural Networks with Self-Attention for Fisheye Image Rectification10.1109/ACCESS.2022.3228297 ↗
38. A real-time correction and stitching algorithm for underwater fisheye images10.1007/s11760-022-02135-0 ↗
39. Yao, X.C., Yu, G.J., Li, G.Q., Yan, S., Zhao, L., and Zhu, D.H. (2023). HexTile: A Hexagonal DGGS-Based Map Tile Algorithm for Visualizing Big Remote Sensing Data in Spark. ISPRS Int. J. Geo-Inf., 12.10.3390/ijgi12030089 ↗
40. A method for stitching remote sensing images with Delaunay triangle feature constraints10.1080/10106049.2023.2285356 ↗
41. Towards an Understanding of Residual Networks Using Neural Tangent Hierarchy (NTH)10.4208/csiam-am.SO-2021-0053 ↗
42. MR Image Super-Resolution via Wide Residual Networks with Fixed Skip Connection10.1109/JBHI.2018.2843819 ↗
43. Chen, Z.Y., Wang, Y.H., Wu, J., Deng, C., and Jiang, W.X. (2022). Wide Residual Relation Network-Based Intelligent Fault Diagnosis of Rotating Machines with Small Samples. Sensors, 22.10.3390/s22114161 ↗
44. Erfurt, J., Helmrich, C.R., Bosse, S., Schwarz, H., Marpe, D., and Wiegand, T. (2019, January 22–25). A study of the perceptually weighted peak signal-to-noise ratio (WPSNR) for image compression. Proceedings of the 26th IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8803307 ↗
45. Seam elimination based on Curvelet for image stitching10.1007/s00500-018-3175-0 ↗
46. PSNR vs. SSIM: Imperceptibility quality assessment for image steganography10.1007/s11042-020-10035-z ↗
47. Li, T., Huang, J., Zhang, Y., and Liu, S. (2019, January 20–21). A Geometric Distortion Rectification Method for Wide Field of View Camera. Proceedings of the International Conference on Electronical, Mechanical and Materials Engineering (ICE2ME), Wuhan, China.10.2991/ice2me-19.2019.7 ↗
48. Liao, Q., Lin, Q., Jin, L., Luo, C., Zhang, J., Peng, D., and Wang, T. (2021, January 5–10). A Multi-level Progressive Rectification Mechanism for Irregular Scene Text Recognition. Proceedings of the 16th IAPR International Conference on Document Analysis and Recognition (ICDAR), Lausanne, Switzerland.10.1007/978-3-030-86337-1_10 ↗
49. Zhang, Y., Wang, T., Zheng, T.J., Zhang, Y.S., Li, L., Yu, Y., and Li, L. (2023). On-Orbit Geometric Calibration and Performance Validation of the GaoFen-14 Stereo Mapping Satellite. Remote Sens., 15.10.3390/rs15174256 ↗
50. Venkatanath, N., Praneeth, D., Bh, M.C., Channappayya, S.S., and Medasani, S.S. (March, January 27). Blind image quality evaluation using perception based features. Proceedings of the 21st National Conference on Communications (NCC), Bombay, India.
51. Making a “Completely Blind” Image Quality Analyzer10.1109/LSP.2012.2227726 ↗
52. Yelmanov, S., and Romanyshyn, Y. (2020, January 22–24). Quantifying the contrast of objects in a complex image. Proceedings of the IEEE 40th International Conference on Electronics and Nanotechnology (ELNANO), Kyiv, Ukraine.10.1109/ELNANO50318.2020.9088760 ↗
53. General temporal ghost imaging model with detection resolution and noise10.1364/AO.479043 ↗
54. Image Completion Based on Edge Prediction and Improved Generator
55. Zhao, P., Hu, Q., Tang, Z., and Ai, M. (2020). A Smooth Transition Algorithm for Adjacent Panoramic Viewpoints Using Matched Delaunay Triangular Patches. ISPRS Int. J. Geo-Inf., 9.10.3390/ijgi9100596 ↗
56. Remote Sensing Image Mosaic Based on Distribution Measure and Saliency Information
57. Wang, Z., Shen, H., and Huang, K. (2023, January 29). Taming Vector-Wise Quantization for Wide-Range Image Blending with Smooth Transition. Proceedings of the 1st International Workshop on Multimedia Content Generation and Evaluation-New Methods and Practice (McGE), Ottawa, ON, Canada.10.1145/3607541.3616809 ↗
58. Stulpinas, R., Morkunas, M., Rasmusson, A., Drachneris, J., Augulis, R., Gulla, A., Strupas, K., and Laurinavicius, A. (2024). Improving HCC Prognostic Models after Liver Resection by AI-Extracted Tissue Fiber Framework Analytics. Cancers, 16.10.3390/cancers16010106 ↗
59. Isotropic discretization methods of Laplacian and generalized divergence operators in phase field models10.1016/j.commatsci.2023.112688 ↗
60. 3D Printed Magneto-Active Microfiber Scaffolds for Remote Stimulation and Guided Organization of 3D In Vitro Skeletal Muscle Models
61. Domain independent post-processing with graph U-nets: Applications to electrical impedance tomographic imaging10.1088/1361-6579/ad0b3d ↗
62. Efficient boundary surface reconstruction from multi-label volumetric data with mathematical morphology10.1016/j.cag.2023.10.003 ↗
63. Levy, T., and May, G. (2024). Conservative solution transfer between anisotropic meshes for time-accurate hybridized discontinuous Galerkin methods. Int. J. Numer. Methods Fluids, 1–20.10.1002/fld.5278 ↗
64. SpecTrHuMS: Spectral transformer for human mesh sequence learning10.1016/j.cag.2023.07.001 ↗