PSRGAN-RK: Real-time infrared super-resolution based on MobileNetV4 and RKNN

Deep learning-based super-resolution (SR) techniques for infrared images have made significant progress. However, most existing algorithms focus on single-image SR using complex neural networks, which are challenging to train, slow to infer, and unsuitable for real-time tasks. Therefore, improving computational efficiency and reducing training costs while maintaining high SR quality remains a critical challenge in edge SR computing. To address this issue, we first propose PSRGAN-Mobile, an improved version based on MobileNetV4, incorporating both main and sub-paths. Within the main path, an inverted residual bottleneck block with channel attention (UIB-SE) extracts mid-to-high-level features from infrared images. Subsequently, lightweight residual blocks and distilled inverted residual bottleneck blocks (RES-UIB) extract shallow-level features from infrared images along the sub-path. Building upon transfer learning principles, we employ a multi-stage transfer learning strategy to enhance feature extraction capabilities across network components in training, thereby improving SR performance. Experimental results on public datasets demonstrate that our model achieves high PSNR (37.86 dB) and SSIM (0.9427) with significantly fewer parameters than other popular SR models, while also exhibiting faster training convergence. Furthermore, to enhance computational efficiency, inspired by RKNN model acceleration for inference, we quantize the model to obtain PSRGAN-RK. Empirical tests reveal that single-image inference for 640 × 640 images on edge devices takes an average time of 257.29 ms, a 94.7% reduction compared to PSRGAN-Mobile, and real-time average frame rates even reach 14 fps. These metrics demonstrate that our method achieves exceptional SR efficiency, making it well-suited for edge computing applications.