Exploring Deep Neural Networks in Simulating Human Vision through Five Optical Illusions
Source abstract
Abstract
Recent research has delved into the biological parallels between deep neural networks (DNNs) in vision and human perception through the study of visual illusions. However, the bulk of this research is currently constrained to the investigation of visual illusions within a single model focusing on a singular type of illusion. There exists a need for a more comprehensive explanation of visual illusions in DNNs, as well as an expansion in the variety of illusions studied. This study is pioneering in its application of representational dissimilarity matrices and feature activation visualization techniques for a detailed examination of how five classic visual illusions are processed by DNNs. Our findings uncover the potential of DNNs to mimic human visual illusions, particularly highlighting notable differences in how these networks process illusions pertaining to color, contrast, length, angle, and spatial positioning. Although there are instances of consistency between DNNs and human perception in certain illusions, the performance distribution and focal points of interest within the models diverge from those of human observers. This study significantly advances our comprehension of DNNs’ capabilities in handling complex visual tasks and their potential to emulate the human biological visual system. It also underscores the existing gaps in our understanding and processing of intricate visual information. While DNNs have shown progress in simulating human vision, their grasp of the nuance and intricacy of complex visual data still requires substantial improvement.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Optical Sensing Technologies · Visual perception and processing mechanisms · Advanced Vision and Imaging
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Graduate School of Engineering, Kochi University of Technology, Kami 782-8502, Kochi, Japan
School of Informatics, Kochi University of Technology, Kami 782-8502, Kochi, Japan
Funding
JSPS KAKENHI
References (46)
Show source-provided references
1. Visual illusions and neurobiology10.1038/35104092 ↗
2. Synaesthesia—A window into perception, thought and language
3. Knowledge in perception and illusion10.1098/rstb.1997.0095 ↗
4. The empirical basis of color perception10.1016/S1053-8100(02)00014-4 ↗
5. Deep learning10.1038/nature14539 ↗
6. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
7. Imagenet classification with deep convolutional neural networks
8. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
9. Illusory motion reproduced by deep neural networks trained for prediction10.3389/fpsyg.2018.00345 ↗
10. Zhang, H., Yoshida, S., and Li, Z. (2023, January 1–4). Decoding Illusion Perception: A Comparative Analysis of Deep Neural Networks in the Müller-Lyer Illusion. Proceedings of the 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Honolulu, HI, USA.10.1109/SMC53992.2023.10394255 ↗
11. ImageNet-trained deep neural networks exhibit illusion-like response to the Scintillating grid10.1167/jov.21.11.15 ↗
12. Zhang, H., Yoshida, S., and Li, Z. (2024). Brain-like illusion produced by Skye’s Oblique Grating in deep neural networks. PLoS ONE, 19.10.1371/journal.pone.0299083 ↗
13. Using goal-driven deep learning models to understand sensory cortex10.1038/nn.4244 ↗
14. Cadieu, C.F., Hong, H., Yamins, D.L., Pinto, N., Ardila, D., Solomon, E.A., Majaj, N.J., and DiCarlo, J.J. (2014). Deep neural networks rival the representation of primate IT cortex for core visual object recognition. PLoS Comput. Biol., 10.10.1371/journal.pcbi.1003963 ↗
15. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
16. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 1). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
17. How does the brain solve visual object recognition?10.1016/j.neuron.2012.01.010 ↗
18. Performance-optimized hierarchical models predict neural responses in higher visual cortex10.1073/pnas.1403112111 ↗
19. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. (2013). Intriguing properties of neural networks. arXiv.
20. Nguyen, A., Yosinski, J., and Clune, J. (2015, January 12). Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298640 ↗
21. Ward, E.J. (2019). Exploring perceptual illusions in deep neural networks. BioRxiv, 19.10.1101/687905 ↗
22. Schrimpf, M., Kubilius, J., Hong, H., Majaj, N.J., Rajalingham, R., Issa, E.B., Kar, K., Bashivan, P., Prescott-Roy, J., and Geiger, F. (2018). Brain-score: Which artificial neural network for object recognition is most brain-like?. BioRxiv, BioRxiv:407007.10.1101/407007 ↗
23. Integrative benchmarking to advance neurally mechanistic models of human intelligence10.1016/j.neuron.2020.07.040 ↗
24. Brain hierarchy score: Which deep neural networks are hierarchically brain-like?10.1016/j.isci.2021.103013 ↗
25. Representational geometry: Integrating cognition, computation, and the brain10.1016/j.tics.2013.06.007 ↗
26. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 20–23). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Cambridge, MA, USA.10.1109/ICCV.2017.74 ↗
27. Chattopadhay, A., Sarkar, A., Howlader, P., and Balasubramanian, V.N. (2018, January 15). Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.10.1109/WACV.2018.00097 ↗
28. The psychophysics toolbox10.1163/156856897X00357 ↗
29. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 11–14). Identity mappings in deep residual networks. Proceedings of the Computer Vision—ECCV 2016: 14th European Conference, Amsterdam, The Netherlands.10.1007/978-3-319-46493-0_38 ↗
30. Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K. (2017, January 26). Aggregated residual transformations for deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.634 ↗
31. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 26). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
32. Wightman, R. (2024, April 05). PyTorch Image Models. Available online: https://github.com/rwightman/pytorch-image-models.
33. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE conference on computer vision and pattern recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
34. Surface color from boundaries: A new ‘watercolor’ illusion10.1016/S0042-6989(01)00105-5 ↗
35. Ecology, perceptual development and the Müller-Lyer illusion10.1111/j.2044-8295.1968.tb01134.x ↗
36. The components of the Poggendorff illusion10.1111/j.2044-8295.1976.tb01545.x ↗
37. Determinants of the Zöllner illusion10.1007/BF00309038 ↗
38. Color illusions also deceive CNNs for low-level vision tasks: Analysis and implications10.1016/j.visres.2020.07.010 ↗
39. Engilberge, M., Collins, E., and Süsstrunk, S. (2017, January 20). Color representation in deep neural networks. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.10.1109/ICIP.2017.8296790 ↗
40. Color for object recognition: Hue and chroma sensitivity in the deep features of convolutional neural networks10.1016/j.visres.2020.09.010 ↗
41. Deep neural networks: A new framework for modeling biological vision and brain information processing10.1146/annurev-vision-082114-035447 ↗
42. The Hermann grid illusion revisited10.1068/p5447 ↗
43. The Poggendorff illusion driven by real and illusory contour: Behavioral and neural mechanisms10.1016/j.neuropsychologia.2016.03.005 ↗
44. Ventral and dorsal stream interactions during the perception of the Müller-Lyer illusion: Evidence derived from fMRI and dynamic causal modeling10.1162/jocn_a_00258 ↗
45. The human visual cortex10.1146/annurev.neuro.27.070203.144220 ↗
46. Polarity-dependent orientation illusions: Review, model, and simulations10.1016/j.visres.2021.09.003 ↗