Evaluating Activation Functions in GAN Models for Virtual Inpainting: A Path to Architectural Heritage Restoration
Source abstract
Abstract
Computer vision has advanced much in recent years. Several tasks, such as image recognition, classification, or image restoration, are regularly solved with applications using artificial intelligence techniques. Image restoration comprises different use cases such as style transferring, improvement of quality resolution, or completing missing parts. The latter is also known as image inpainting, virtual image inpainting in this case, which consists of reconstructing missing regions or elements. This paper explores how to evaluate the performance of a deep learning method to do virtual image inpainting to reconstruct missing architectonical elements in images of ruined Greek temples to measure the performance of different activation functions. Unlike a previous study related to this work, a direct reconstruction process without segmented images was used. Then, two evaluation methods are presented: the objective one (mathematical metrics) and an expert (visual perception) evaluation to measure the performance of the different approaches. Results conclude that ReLU outperforms other activation functions, while Mish and Leaky ReLU perform poorly, and Swish’s professional evaluations highlight a gap between mathematical metrics and human visual perception.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Computer Graphics and Visualization Techniques · Advanced Image Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
CEIEC Research Institute, Universidad Francisco de Vitoria, Ctra. M-515 Pozuelo-Majadahonda km. 1, 800, Pozuelo de Alarcón, 28223 Madrid, Spain
Architecture School, Universidad Francisco de Vitoria, Ctra. M-515 Pozuelo-Majadahonda km. 1, 800, Pozuelo de Alarcón, 28223 Madrid, Spain
Funding
MICIU/AEI/10.13039/501100011033 · FEDER, UE
References (38)
Show source-provided references
1. Richmond, A., and Bracker, A. (2009). The Reconstruction of Ruins: Principles and Practice. Conservation: Principles, Dilemmas and Uncomfortable Truths, Elsevier.
2. Deep learning10.1038/nature14539 ↗
3. Nogales, A., Delgado-Martos, E., Melchor, Á., and García-Tejedor, Á.J. (2021). ARQGAN: An evaluation of generative adversarial network approaches for automatic virtual inpainting restoration of Greek temples. Expert Syst. Appl., 180.10.1016/j.eswa.2021.115092 ↗
4. Digital Restoration of Cultural Heritage with Data-Driven Computing: A Survey10.1109/ACCESS.2023.3280639 ↗
5. Data-driven conservation actions of heritage places curated with HBIM10.4995/var.2022.17370 ↗
6. Andriasyan, M., Moyano, J., Nieto-Julián, J.E., and Antón, D. (2020). From Point Cloud Data to Building Information Modelling: An Automatic Parametric Workflow for Heritage. Remote Sens., 12.10.3390/rs12071094 ↗
7. Deep learning—Based image restoration algorithm for coronary CT angiography10.1007/s00330-019-06183-y ↗
8. Fiber bundle image restoration using deep learning10.1364/OL.44.001080 ↗
9. StatNet: Statistical image restoration for low-dose CT using deep learning10.1109/JSTSP.2020.2998413 ↗
10. Jiao, J., Tu, W.-C., He, S., and Lau, R.W.H. (2017, January 21–26). Formresnet: Formatted residual learning for image restoration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.140 ↗
11. SMukherjee, S., Dittmer, S., Shumaylov, Z., Lunz, S., Öktem, O., and Schönlieb, C.B. (2024, January 14–19). Data-Driven Convex Regularizers for Inverse Problems. Proceedings of the ICASSP 2024—2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10447719 ↗
12. End-to-end reconstruction meets data-driven regularization for inverse problems
13. Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., and Li, H. (2022, January 18–24). Uformer: A general u-shaped transformer for image restoration. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01716 ↗
14. Artificial intelligence-assisted visual inspection for cultural heritage: State-of-the-art review10.1016/j.culher.2024.01.005 ↗
15. Generative Adversarial Networks for Single Photo 3D Reconstruction10.5194/isprs-archives-XLII-2-W9-403-2019 ↗
16. Yu, C. (2019, January 10–16). Semi-supervised three-dimensional reconstruction framework with GAN. Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China.10.24963/ijcai.2019/582 ↗
17. Theodorus, A. (2020). Restoration of Damaged Face Statues Using Deep Generative Inpainting Model. [Master’s Thesis, University of Twente].
18. Abed, M.H., Al-Asfoor, M., and Hussain, Z.M. (2020, January 30–31). Architectural heritage images classification using deep learning with CNN. Proceedings of the 2nd International Workshop on Visual Pattern Extraction and Recognition for Cultural Heritage Understandingco-Located with 16th Italian Research Conference on Digital Libraries (IRCDL 2020), Bari, Italy.
19. Reis, H.C., and Khoshelham, K. (2021). ReCRNet: A deep residual network for crack detection in historical buildings. Arab. J. Geosci., 14.10.1007/s12517-021-08491-4 ↗
20. Automatic damage detection of stone cultural property based on deep learning algorithm10.5194/isprs-archives-XLII-2-W15-639-2019 ↗
21. Artificial intelligence-based visual inspection system for structural health monitoring of cultural heritage10.1007/s13349-022-00643-8 ↗
22. Applying Deep Learning and Convolutional Neural Network System to Identify Historic Buildings: The ‘Little China’ Building in Central Java, Indonesia
23. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. Adv. Neural Inf. Process. Syst., 27.
24. Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A.A. (2017, January 21–26). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.632 ↗
25. Reducing the dimensionality of data with neural networks10.1126/science.1127647 ↗
26. Szandała, T. (2021). Review and comparison of commonly used activation functions for deep neural networks. Bio-Inspired Neurocomputing, Springer.10.1007/978-981-15-5495-7_11 ↗
27. Gulcehre, C., Moczulski, M., Denil, M., and Bengio, Y. (2016, January 20–22). Noisy activation functions. Proceedings of the International Conference on Machine Learning, New York, NY, USA.
28. Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit10.1038/35016072 ↗
29. Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013, January 16–21). Rectifier nonlinearities improve neural network acoustic models. Proceedings of the International Conference on Machine Learning (ICML 2013), Atlanta, GA, USA.
30. Ramachandran, P., Zoph, B., and Le, Q.V. (2017). Searching for activation functions. arXiv.
31. Misra, D. (2019). Mish: A self regularized non-monotonic neural activation function. arXiv.
32. Algorithms for hyper-parameter optimization
33. Ruder, S. (2016). An overview of gradient descent optimization algorithms. arXiv.
34. Ioffe, S., and Szegedy, C. (2015, January 6–11). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning, Lille, France.
35. Ulyanov, D., Vedaldi, A., and Lempitsky, V. (2016). Instance Normalization: The Missing Ingredient for Fast Stylization. arXiv.
36. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
37. Rmsprop: Divide the gradient by a running average of its recent magnitude
38. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.