Communication in Human–AI Co-Creation: Perceptual Analysis of Paintings Generated by Text-to-Image System
Source abstract
Abstract
In recent years, art creation using artificial intelligence (AI) has started to become a mainstream phenomenon. One of the latest applications of AI is to generate visual artwork from natural language descriptions where anyone can interact with it to create thousands of artistic images with minimal effort, which provokes the questions: what is the essence of artistic creation, and who can create art in this era? Considering that, in this study, the theoretical communication framework was adopted to investigate the difference in the interaction with the text-to-image system between artists and nonartists. In this experiment, ten artists and ten nonartists were invited to co-create with Midjourney. Their actions and reflections were recorded, and two sets of generated images were collected for the visual question-answering task, with a painting created by the artist as a reference sample. A total of forty-two subjects with artistic backgrounds participated in the evaluated experiment. The results indicated differences between the two groups in their creation actions and their attitude toward AI, while the technology blurred the difference in the perception of the results caused by the creator’s artistic experience. In addition, attention should be paid to communication on the effectiveness level for a better perception of the artistic value.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Aesthetic Perception and Analysis · Visual Culture and Art Theory · Virtual Reality Applications and Impacts
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Digital Media Arts, School of Media and Design, Beijing Technology and Business University, Beijing 102488, China
Key Lab of Encyclopedia Knowledge Fusion Innovation Publishing Project, Beijing 100037, China
Art Teaching and Research Section, Beijing International Studies University, Beijing 100024, China
Graduate School of Creative Industry Design, National Taiwan University of Arts, New Taipei 220307, Taiwan
Department of Digital Media Arts, School of Art and Design, Shenzhen University, Shenzhen 518061, China
Funding
Beijing Municipal Education Commission
References (49)
Show source-provided references
1. Creative processes in painting10.1080/00221309.1952.9710660 ↗
2. The figural problem solving and problem finding of professional and semiprofessional artists and nonartists10.1080/10400419109534396 ↗
3. (2022, June 10). Disco Diffusion. Available online: https://github.com/alembics/disco-diffusion.
4. (2022, August 25). Midjourney. Available online: www.midjourney.com.
5. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022, January 18–24). High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01042 ↗
6. Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. (2022). Hierarchical text-conditional image generation with clip latents. arXiv.
7. Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S.K.S., Ayan, B.K., Mahdavi, S.S., and Lopes, R.G. (2022). Photorealistic Text.-to-Image Diffusion Models with Deep Language Understanding. arXiv.10.1145/3528233.3530757 ↗
8. (2022, August 25). State Fair’s Website. Available online: https://coloradostatefair.com/wp-content/uploads/2022/08/2022-Fine-Arts-First-Second-Third.pdf.
9. Gu, S., Chen, D., Bao, J., Wen, F., Zhang, B., Chen, D., Yuan, L., and Guo, B. (2022, January 18–24). Vector quantized diffusion model for text-to-image synthesis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01043 ↗
10. Crowson, K., Biderman, S., Kornis, D., Stander, D., Hallahan, E., Castricato, L., and Raff, E. (2022, January 23–27). Vqgan-clip: Open domain image generation and editing with natural language guidance. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-19836-6_6 ↗
11. Lee, H., Ullah, U., Lee, J.S., Jeong, B., and Choi, H.C. (2021, January 1–3). A Brief Survey of text driven image generation and maniulation. Proceedings of the 2021 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), Gangneung, Korea.10.1109/ICCE-Asia53811.2021.9641929 ↗
12. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., and Goh, G. (2021, January 18–24). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning, Virtual Event.
13. Learning to prompt for vision-language models10.1007/s11263-022-01653-1 ↗
14. Liu, V., and Chilton, L.B. (May, January 30). Design Guidelines for Prompt Engineering Text-to-Image Generative Models. Proceedings of the CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA.
15. Wu, Y., Yu, N., Li, Z., Backes, M., and Zhang, Y. (2022). Membership Inference Attacks Against Text-to-image Generation Models. arXiv.
16. Van Den Oord, A., and Vinyals, O. (2017, January 4–9). Neural discrete representation learning. Proceedings of the Neural Information Processing Systems Annual Conference, Long Beach, CA, USA.
17. Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., and Sutskever, I. (2021, January 18–24). Zero-shot text-to-image generation. Proceedings of the International Conference on Machine Learning, Virtual Event.
18. Yu, J., Xu, Y., Koh, J.Y., Luong, T., Baid, G., Wang, Z., Vasudevan, V., Ku, A., Yang, Y., and Ayan, B.K. (2022). Scaling autoregressive models for content-rich text-to-image generation. arXiv.
19. Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. (2015, January 6–11). Deep unsupervised learning using nonequilibrium thermodynamics. Proceedings of the International Conference on Machine Learning, Lille, France.
20. (2022, September 02). Stable-Diffusion. Available online: https://github.com/CompVis/stable-diffusion.
21. Understanding and creating art with AI: Review and outlook10.1145/3475799 ↗
22. The cognition of turning poetry into painting
23. Audry, S. (2021). Art in the Age of Machine Learning, MIT Press.10.7551/mitpress/12832.001.0001 ↗
24. Solso, R.L. (1996). Cognition and the Visual Arts, MIT Press.
25. Visual Aesthetic Experience10.2307/4140196 ↗
26. Uncovering the connection between artist and audience: Viewing painted brushstrokes evokes corresponding action representations in the observer10.1016/j.cognition.2012.06.012 ↗
27. Gombrich, Galenson, and beyond: Integrating case study and typological frameworks in the study of creative individuals10.2190/EM.26.1.e ↗
28. Ericsson, K.A., Hoffman, R.R., and Kozbelt, A. (2018). Expertise in drawing. The Cambridge Handbook of Expertise and Expert Performance, Cambridge University Press.10.1017/9781316480748 ↗
29. Investigating the negative bias towards artificial intelligence: Effects of prior assignment of AI-authorship on the aesthetic appreciation of abstract paintings10.1016/j.chb.2022.107406 ↗
30. Lyu, Y. (2022, October 23). A Study on Perception of Artistic Style Tansfer using Artificial Intelligance Technology. Unpublished Doctor’s Thesis, National Taiwan University, Taipei, Taiwan. Available online: https://hdl.handle.net/11296/grdz93.
31. Lyu, Y., Lin, C.-L., Lin, P.-H., and Lin, R. (2021). The Cognition of Audience to Artistic Style Transfer. Appl. Sci., 11.10.3390/app11073290 ↗
32. Sun, Y., Yang, C.H., Lyu, Y., and Lin, R. (2022). From Pigments to Pixels: A Comparison of Human and AI Painting. Appl. Sci., 12.10.3390/app12083724 ↗
33. Fiske, J. (2010). Introduction to Communication Studies, Routledge. [3rd ed.].10.4324/9780203837382 ↗
34. Jakobson, R. (1987). Language in literature, Harvard University Press.
35. Lin, R., Qian, F., Wu, J., Fang, W.-T., and Jin, Y. (2017, January 9–14). A Pilot Study of Communication Matrix for Evaluating Artworks. Proceedings of the International Conference on Cross-Cultural Design, Vancouver, BC, Canada.10.1007/978-3-319-57931-3_29 ↗
36. Mazzone, M., and Elgammal, A. (2019). Art, creativity, and the potential of artificial intelligence. Arts, 8.10.3390/arts8010026 ↗
37. Gao, Y.-J., Chen, L.-Y., Lee, S., Lin, R., and Jin, Y. (2017, January 9–14). A study of communication in turning “poetry” into “painting”. Proceedings of the International Conference on Cross-Cultural Design, Vancouver, BC, Canada.10.1007/978-3-319-57931-3_4 ↗
38. Gao, Y., Wu, J., Lee, S., and Lin, R. (2019, January 26–31). Communication Between Artist and Audience: A Case Study of Creation Journey. Proceedings of the International Conference on Human-Computer Interaction, Orlando, FL, USA.10.1007/978-3-030-22580-3_3 ↗
39. Yu, Y., Binghong, Z., Fei, G., and Jiaxin, T. (2020, January 10–13). Research on Artificial Intelligence in the Field of Art Design Under the Background of Convergence Media. Proceedings of the IOP Conference Series: Materials Science and Engineering, Ulaanbaatar, Mongolia.10.1088/1757-899X/825/1/012027 ↗
40. (2022, August 25). Promptbase. Available online: https://promptbase.com/.
41. Hageback, N., and Hedblom, D. (2021). AI FOR ARTS, CRC Press.10.1201/9781003195009 ↗
42. Hertzmann, A. (2018). Can Computers Create Art?. Arts, 7.10.3390/arts7020018 ↗
43. Oppenlaender, J. (2022). Prompt Engineering for Text-Based Generative Art. arXiv.
44. Ghosh, A., and Fossas, G. (2022). Can There be Art Without an Artist?. arXiv.
45. Putting the art in artificial: Aesthetic responses to computer-generated art10.1037/aca0000136 ↗
46. Artificial intelligence, artists, and art: Attitudes toward artwork produced by humans vs. artificial intelligence
47. The role of AI attribution knowledge in the evaluation of artwork10.1177/0276237421994697 ↗
48. Corbin, J., and Strauss, A. (1998). Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Sage Publications.
49. Lin, Z.Y. (2007). Multivariate Analysis, Best-Wise Publishing Co., Ltd.