Transmission Removal from a Single OSN-Shared Glass Mixture Image
Source abstract
Abstract
Photographs taken through glass often reflect the photographer or the surroundings, which is very helpful in uncovering information about the photograph. Various lossy operations performed on images over online social networks (OSNs), such as compression and resampling, pose a great challenge for transmission layer removal. This paper proposes a self-attention-based architecture for image enhancement over OSNs, to ensure that the downloaded glass mixture image can show more information about the reflection layer than the original image. Transmission layer removal is then achieved using a two-stage generative adversarial network. We also add attention to the transmission layer in the mixture image and use the gradient and color block information in the next stage to extract the reflection layer. This method yielded a gain of 0.46 dB in PSNR, 0.016 in SSIM, and 0.057 in LPIPS, resulting in an effective improvement in the visual quality of the final extracted reflection layer images.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Processing Techniques · Digital Media Forensic Detection · Image and Signal Denoising Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
Funding
Natural Science Foundation of China · Natural Science Foundation of Shanghai · STCSM Capability Construction Project for Shanghai Municipal Universities
References (31)
Show source-provided references
1. Ra, M.R., Govindan, R., and Ortega, A. (2013, January 2–5). P3: Toward {Privacy-Preserving} Photo Sharing. Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 13), Lombard, IL, USA.
2. Ning, J., Singh, I., Madhyastha, H.V., Krishnamurthy, S.V., Cao, G., and Mohapatra, P. (2014, January 29–31). Secret message sharing using online social media. Proceedings of the IEEE Conference on Communications and Network Security, San Francisco, CA, USA.10.1109/CNS.2014.6997500 ↗
3. Sun, W., Zhou, J., Lyu, R., and Zhu, S. (2016, January 15–19). Processing-aware privacy-preserving photo sharing over online social networks. Proceedings of the 24th ACM international conference on Multimedia, Amsterdam, The Netherlands.10.1145/2964284.2967288 ↗
4. Face image reflection removal10.1007/s11263-020-01372-5 ↗
5. Image reflection removal using end-to-end convolutional neural network10.1049/iet-ipr.2019.0247 ↗
6. Joint reflection removal and depth estimation from a single image10.1109/TCYB.2019.2959381 ↗
7. Li, C., Yang, Y., He, K., Lin, S., and Hopcroft, J.E. (2020, January 13–19). Single image reflection removal through cascaded refinement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00362 ↗
8. User assisted separation of reflections from a single image using a sparsity prior10.1109/TPAMI.2007.1106 ↗
9. Zhang, X., Ng, R., and Chen, Q. (2018, January 18–23). Single image reflection separation with perceptual losses. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00503 ↗
10. CoRRN: Cooperative reflection removal network10.1109/TPAMI.2019.2921574 ↗
11. Wan, R., Shi, B., Duan, L.Y., Tan, A.H., and Kot, A.C. (2018, January 18–23). CRRN: Multi-scale guided concurrent reflection removal network. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00502 ↗
12. Li, Z., Yao, H., Shi, R., Qiao, T., and Qin, C. (2022). Transmission Removal from a Single Glass Scene and Its Application in Photographer Identification. Appl. Sci., 12.10.3390/app122312484 ↗
13. Robust image-adaptive data hiding using erasure and error correction10.1109/TIP.2004.837557 ↗
14. Digital watermarking robust to geometric distortions10.1109/TIP.2005.857263 ↗
15. Image watermarking based on invariant regions of scale-space representation10.1109/TSP.2006.870581 ↗
16. Perceptual DFT watermarking with improved detection and robustness to geometrical distortions10.1109/TIFS.2014.2322497 ↗
17. A new digital image watermarking algorithm resilient to desynchronization attacks10.1109/TIFS.2007.908233 ↗
18. Rotation, scale, and translation resilient watermarking for images10.1109/83.918569 ↗
19. Robust image forgery detection against transmission over online social networks10.1109/TIFS.2022.3144878 ↗
20. Wu, Y., AbdAlmageed, W., and Natarajan, P. (2019, January 15–20). Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00977 ↗
21. Exposing fake images with forensic similarity graphs10.1109/JSTSP.2020.3001516 ↗
22. Attention is all you need
23. Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., and Li, H. (2022, January 18–24). Uformer: A general u-shaped transformer for image restoration. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01716 ↗
24. SwinF: Swin Transformer with feature fusion in target detection
25. Perceptual loss-constrained adversarial autoencoder networks for hyperspectral unmixing
26. Mechrez, R., Talmi, I., Shama, F., and Zelnik-Manor, L. (2019). Proceedings of the 14th Asian Conference on Computer Vision, Springer.
27. Lei, C., and Chen, Q. (2021, January 20–25). Robust reflection removal with reflection-free flash-only cues. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01457 ↗
28. Li, Y., and Brown, M.S. (2014, January 23–28). Single image layer separation using relative smoothness. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.346 ↗
29. Chang, Y.C., Lu, C.N., Cheng, C.C., and Chiu, W.C. (2021, January 5–9). Single image reflection removal with edge guidance, reflection classifier, and recurrent decomposition. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Virtual.10.1109/WACV48630.2021.00208 ↗
30. Yang, J., Gong, D., Liu, L., and Shi, Q. (2018, January 8–14). Seeing deeply and bidirectionally: A deep learning approach for single image reflection removal. Proceedings of the the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01219-9_40 ↗
31. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–23). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00068 ↗