Human–AI Co-Drawing: Studying Creative Efficacy and Eye Tracking in Observation and Cooperation
Source abstract
Abstract
Artificial intelligence (AI) tools are rapidly transforming the field of traditional artistic creation, influencing painting processes and human creativity. This study explores human–AI cooperation in real-time artistic drawing by using the AIGC tool KREA.AI. Participants wear eye trackers and perform drawing tasks by adjusting the AI parameters. The research aims to investigate the impact of cross-screen and non-cross-screen conditions, as well as different viewing strategies, on cognitive load and the degree of creative stimulation during user–AI collaborative drawing. Adopting a mixed design, it examines the influence of different cooperation modes and visual search methods on creative efficacy and visual perception through eye-tracking data and creativity performance scales. The cross-screen type and task type have a significant impact on total interval duration, number of fixation points, average fixation duration, and average pupil diameter in occlusion decision-making and occlusion hand drawing. There are significant differences in the variables of average gaze duration and average pupil diameter among different task types and cross-screen types. In non-cross-screen situations, occlusion and non-occlusion have a significant impact on average gaze duration and pupil diameter. Tasks in non-cross-screen environments are more sensitive to visual processing. The involvement of AI in hand drawing in non-cross-screen collaborative drawing by designers has a significant impact on their visual perception. These results help us to gain a deeper understanding of user behaviour and cognitive load under different visual tasks and cross-screen conditions. The analysis of the creative efficiency scale data reveals significant differences in designers’ ability to supplement and improve AI ideas across different modes. This indicates that the extent of AI participation in the designer’s hand-drawn creative process significantly impacts the designer’s behaviour when negotiating design ideas with the AI.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Virtual Reality Applications and Impacts · Creativity in Education and Neuroscience · Design Education and Practice
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechanical Engineering, Southeast University, Nanjing 211189, China
Funding
Research on Human–Computer Interaction Interface Design Mechanism for Human–Computer Collaboration, National Natural Science Foundation of China
References (62)
Show source-provided references
1. Lyu, Y., Wang, X., Lin, R., and Wu, J. (2022). Communication in Human–AI Co-Creation: Perceptual Analysis of Paintings Generated by Text-to-Image System. Appl. Sci., 12.10.3390/app122211312 ↗
2. Artificial Intelligence-Based Interactive Art Design under Neural Network Vision Valve10.1155/2022/3628955 ↗
3. Artificial Intelligence-Generated Content in Intelligent Transportation Systems: Learning to Copy, Change, and Create! [Editor’s Column]10.1109/MITS.2023.3295392 ↗
4. (2024, January 20). AIGC: The Rise and Future Prospects of Artificial Intelligence Generated Content—Cloud Community—Huawei Cloud. Available online: https://bbs.huaweicloud.com/blogs/398164?utm_source=zhihu&utm_medium=bbs-ex&utm_campaign=other&utm_content=content.
5. Bias in Perception of Art Produced by Artificial Intelligence
6. Redefining Creativity in the Era of AI? Perspectives of Computer Scientists and New Media Artists
7. SmartPaint: A Co-Creative Drawing System Based on Generative Adversarial Networks10.1631/FITEE.1900386 ↗
8. Artificial Intelligence in Fine Arts: A Systematic Review of Empirical Research10.1016/j.chbah.2023.100004 ↗
9. Eye Movements as a Window into Decision-Making10.1146/annurev-vision-100720-125029 ↗
10. Using an AI Creativity System to Explore How Aesthetic Experiences Are Processed along the Brain’s Perceptual Neural Pathways10.1016/j.cogsys.2019.09.012 ↗
11. Divergent Thinking and Evaluative Skill: A Meta-Analysis10.1002/jocb.539 ↗
12. Déguernel, K., and Sturm, B.L.T. (2024, January 20). Bias in Favour or Against Computational Creativity: A Survey and Reflection on the Importance of Socio-Cultural Context in Its Evaluation. Available online: http://kth.diva-portal.org/smash/get/diva2:1757836/FULLTEXT01.pdf.
13. Creativity Self Assessments in Design Education: A Systematic Review10.1016/j.tsc.2024.101494 ↗
14. Creative Analytics: Towards Data-Inspired Creative Decisions10.1111/isj.12369 ↗
15. Task Preparation as a Mnemonic: The Benefits of Drawing (and Not Drawing)10.3758/s13423-018-1477-y ↗
16. Finger versus Pencil: An Eye Tracking Study of Learning by Drawing on Touchscreens10.1111/jcal.12863 ↗
17. Kurosu, M., Hashizume, A., Marcus, A., Rosenzweig, E., Soares, M.M., Harris, D., Li, W.-C., Schmorrow, D.D., Fidopiastis, C.M., and Rau, P.-L.P. (2023, January 23–28). Research on the Efficiency and Cognition of the Combination of Front Color and Background Color and Color in the Interface of Express Cabinets on the Operation of Human Machine Interface Tasks. Proceedings of the HCI International 2023—Late Breaking Papers, Copenhagen, Denmark.
18. A Systematic Review of Eye Tracking Research on Multimedia Learning10.1016/j.compedu.2018.06.023 ↗
19. Blind Image Quality Assessment by Visual Neuron Matrix10.1109/LSP.2021.3106579 ↗
20. Karran, A.J., Demazure, T., Hudon, A., Senecal, S., and Léger, P.-M. (2022). Designing for Confidence: The Impact of Visualizing Artificial Intelligence Decisions. Front. Neurosci., 16.10.3389/fnins.2022.883385 ↗
21. Die Zeichnung im Blick—Nutzung von Eye-Tracking zur Analyse der zeichnerischen Erschließung von Mesomerie-Aufgaben10.1002/ckon.202200007 ↗
22. Cortical Correlates of Creative Thinking Assessed by the Figural Torrance Test of Creative Thinking10.1097/WNR.0000000000001358 ↗
23. Bandi, A., Adapa, P.V.S.R., and Kuchi, Y.E.V.P.K. (2023). The Power of Generative AI: A Review of Requirements, Models, Input–Output Formats, Evaluation Metrics, and Challenges. Future Internet, 15.10.3390/fi15080260 ↗
24. Individuals with Highly Superior Autobiographical Memory Do Not Show Enhanced Creative Thinking10.1080/09658211.2022.2094416 ↗
25. Employee Creativity: Personal and Contextual Factors at Work10.2307/256657 ↗
26. Lee, Y.K., Park, Y.-H., and Hahn, S. (2023). A Portrait of Emotion: Empowering Self-Expression through AI-Generated Art. arXiv.
27. Intelligent Layout Generation Based on Deep Generative Models: A Comprehensive Survey10.1016/j.inffus.2023.101940 ↗
28. Fan, M., Yang, X., Yu, T.T., Liao, V.Q., and Zhao, J. (2021). Human-AI Collaboration for UX Evaluation: Effects of Explanation and Synchronization. arXiv.
29. Kuang, E., Jahangirzadeh Soure, E., Fan, M., Zhao, J., and Shinohara, K. (2023, January 23–28). Collaboration with Conversational AI Assistants for UX Evaluation: Questions and How to Ask Them (Voice vs. Text). Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany.10.1145/3544548.3581247 ↗
30. Cobots in Knowledge Work: Human—AI Collaboration in Managerial Professions10.1016/j.jbusres.2020.11.038 ↗
31. Insights From “The Machine Stops” to Better Understand Rational Assumptions in Algorithmic Decision Making and Its Implications for Organizations10.5465/amr.2018.0181 ↗
32. Six Human-Centered Artificial Intelligence Grand Challenges10.1080/10447318.2022.2153320 ↗
33. Social Integration of Artificial Intelligence: Functions, Automation Allocation Logic and Human-Autonomy Trust10.1007/s12559-018-9619-0 ↗
34. FMASketch: Freehand Mid-Air Sketching in AR10.1080/10447318.2023.2223948 ↗
35. Students’ Collaborative Creative Process and Its Phases in Mathematics: An Explorative Study Using Dual Eye Tracking and Stimulated Recall Interviews10.1007/s11858-022-01327-9 ↗
36. Exploring Students’ Continuance Intention Toward Digital Visual Collaborative Learning Technology in Design Thinking10.1080/10447318.2023.2172904 ↗
37. The Role of Mental Models in Collaborative Sketching10.1007/s10798-014-9269-9 ↗
38. Global Saccadic Eye Movements Characterise Artists’ Visual Attention While Drawing10.1177/02762374211001811 ↗
39. Pupil Size Variations Reveal Information About Hierarchical Decision-Making Processes10.1007/s12559-024-10246-8 ↗
40. Experimental study on icon search characteristics based on feature inference
41. Visual Search Patterns for Multilingual Word Search Puzzles, a Pilot Study10.16910/jemr.16.1.6 ↗
42. Eye Movement Parameters for Performance Evaluation in Projection-Based Stereoscopic Display10.16910/jemr.11.6.3 ↗
43. A Pupillary Index of Susceptibility to Decision Biases10.1038/s41562-020-01006-3 ↗
44. Wu, Z. (2022). Research on Color Emotion Evaluation of AI Images from the Perspective of Sensory Engineering. [Master’s Thesis, Beijing University of Posts and Telecommunications].
45. Evaluating Visual Consistency of Icon Usage in Across-Devices10.1080/10447318.2022.2162275 ↗
46. Study on Error-Cognition Mechanism of Task Interface in Complex Information System10.1007/978-3-319-60525-8_51 ↗
47. CHAP: Open-Source Software for Processing and Analyzing Pupillometry Data10.3758/s13428-018-01190-1 ↗
48. Designing Drawing Apps for Children: Artistic and Technological Factors10.1080/10447318.2021.1926113 ↗
49. Social Presence and Imagery Processing as Predictors of Chatbot Continuance Intention in Human-AI-Interaction10.1080/10447318.2022.2129277 ↗
50. Zhu, M., Bao, D., Yu, Y., Shen, D., and Yi, M. (2022). Differences in Thinking Flexibility between Novices and Experts Based on Eye Tracking. PLoS ONE, 17.10.1371/journal.pone.0269363 ↗
51. Awano, N., and Hayashi, Y. (2022). Object Categorization Capability of Psychological Potential Field in Perceptual Assessment Using Line-Drawing Images. J. Imaging, 8.10.3390/jimaging8040090 ↗
52. I See, You Design: User Interface Intelligent Design System with Eye Tracking and Interactive Genetic Algorithm10.1007/s42486-019-00019-w ↗
53. Bringing Together Ergonomic Concepts and Cognitive Mechanisms for Human—AI Agents Cooperation10.1080/10447318.2022.2129741 ↗
54. What If Artificial Intelligence Become Completely Ambient in Our Daily Lives? Exploring Future Human-AI Interaction through High Fidelity Illustrations10.1080/10447318.2022.2080155 ↗
55. Assessment Model for Perceived Visual Complexity of Painting Images10.1016/j.knosys.2018.06.006 ↗
56. Karimi, P., Rezwana, J., Siddiqui, S., Maher, M.L., and Dehbozorgi, N. (2020, January 17). Creative Sketching Partner: An Analysis of Human-AI Co-Creativity. Proceedings of the 25th International Conference on Intelligent User Interfaces, Cagliari, Italy.10.1145/3377325.3377522 ↗
57. Influence of Personal Innovativeness and Different Sequences of Data Presentation on Evaluations of Explainable Artificial Intelligence10.1080/10447318.2023.2209995 ↗
58. Bozkir, E., Özdel, S., Wang, M., David-John, B., Gao, H., Butler, K., Jain, E., and Kasneci, E. (2024, May 01). Eye-Tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges. Available online: https://arxiv.org/abs/2305.14080v1.
59. Chang, Y., He, C., Zhao, Y., Lu, T., and Gu, N. (2021, January 6–11). A High-Frame-Rate Eye-Tracking Framework for Mobile Devices. Proceedings of the ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9414624 ↗
60. Investigating Visual Expertise in Sculpture: A Methodological Approach Using Eye Tracking10.16910/jemr.15.2.5 ↗
61. Turkmen, R., Pfeuffer, K., Machuca, M.D.B., Batmaz, A.U., and Gellersen, H. (May, January 29). Exploring Discrete Drawing Guides to Assist Users in Accurate Mid-Air Sketching in VR. Proceedings of the Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, CHI 2022, New Orleans, LA, USA.10.1145/3491101.3519737 ↗
62. The Decision Hedgehog for Creative Decision Making10.1007/s10257-007-0067-8 ↗