Extroversion–Introversion Rescheduler in Generative Agent via Few-Shot Prompting
Source abstract
Abstract
Generative Agent (GA) has emerged as a promising framework for simulating human-like behaviors. However, it is required for GA to generate a schedule that consistently reflects the agent’s E-I trait particularly in the extroversion–introversion (E-I) category to improve the realism of GA. We propose an E-I evaluation and rescheduling method that adjusts the agent’s schedule. Specifically, our method takes as input a one-hour schedule segmented into five-minute tasks and a corresponding E-I trait classified into seven degrees ranging from extremely high extroversion to extremely high introversion. Using the Evaluator powered by GPT-4o mini, each task is assessed for the alignment with the E-I traits. Each task that fails to meet a threshold is regenerated using few-shot prompting based on a collected successful schedule. This process is repeated until all tasks are aligned with the corresponding traits. Finally, the evaluator accesses the overall E-I consistency of the schedule that contains the tasks. Therefore, it is possible for the proposed method to enable E-I-consistent schedule generation in GA without retraining any models. In experiments, the proposed framework improved E-I alignment from an average of 14.7% to that of 78.4% with only 1.38 iterations on average, demonstrating both practical effectiveness and computational efficiency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Motion and Animation · Artificial Intelligence in Games · Social Robot Interaction and HRI
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Computer Science and Engineering, Dongguk University-Seoul, Seoul 04620, Republic of Korea
Department of Computer Science and Artificial Intelligence, Dongguk University-Seoul, Seoul 04620, Republic of Korea
Funding
Korea governmen · Seoul RISE Center
References (35)
Show source-provided references
1. Park, J.S., O’Brien, J.C., Cai, C.J., Morris, M.R., Liang, P., and Bernstein, M.S. (November, January 29). Generative Agents: Interactive Simulacra of Human Behavior. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST ’23), Francisco CA, USA.
2. Controlling user perceptions of linguistic style: Trainable generation of personality traits10.1162/COLI_a_00063 ↗
3. Linguistic styles: Language use as an individual difference10.1037/0022-3514.77.6.1296 ↗
4. Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., and Yang, D. (2023). Can large language models transform computational social science?. arXiv.10.1162/coli_a_00502 ↗
5. The Revised NEO Personality Inventory (NEO-PI-R)
6. Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., and Yang, Y. (2023). Self-Refine: Iterative Refinement with Self-Feedback. arXiv.
7. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020, January 6–12). Language Models are Few-Shot Learners. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS 33), Virtual.
8. Aher, G.V., Arriaga, R.I., and Kalai, A.T. (2023, January 23–29). Using large language models to simulate multiple humans and replicate human subject studies. Proceedings of the 40th International Conference on Machine Learning (ICML). PMLR, Honolulu, HI, USA.
9. Andreas, J. (2022). Language models as agent models. arXiv.10.18653/v1/2022.findings-emnlp.423 ↗
10. Ganesan, A.V., Lal, Y.K., Nilsson, A.H., and Schwartz, H.A. (2023). Systematic evaluation of GPT-3 for zero-shot personality estimation. arXiv.10.18653/v1/2023.wassa-1.34 ↗
11. Jiang, G., Xu, M., Zhu, S.C., Han, W., Zhang, C., and Zhu, Y. (2022, January 7–11). Evaluating and inducing personality in pre-trained language models. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, United Arab Emirates.
12. Jiang, G., Xu, M., Zhu, S.C., Han, W., Zhang, C., and Zhu, Y. (2022). MPI: Evaluating and inducing personality in pre-trained language models. arXiv.
13. Li, T., Zheng, X., and Huang, X. (2023). Tailoring personality traits in large language models via unsupervisedly-built personalized lexicons. arXiv.
14. Mao, S., Zhang, N., Wang, X., Wang, M., Yao, Y., Jiang, Y., Xie, P., Huang, F., and Chen, H. (2023). Editing personality for LLMs. arXiv.
15. Shao, Y., Li, L., Dai, J., and Qiu, X. (2023). Character-LLM: A trainable agent for role-playing. arXiv.10.18653/v1/2023.emnlp-main.814 ↗
16. Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., and Weston, J. (2018). Personalizing dialogue agents: I have a dog, do you have pets too?. arXiv.10.18653/v1/P18-1205 ↗
17. Yang, T., Shi, T., Wan, F., Quan, X., Wang, Q., Wu, B., and Wu, J. (2023). PsyCoT: Psychological questionnaire as powerful chain-of-thought for personality detection. arXiv.10.18653/v1/2023.findings-emnlp.216 ↗
18. The structure of phenotypic personality traits10.1037/0003-066X.48.1.26 ↗
19. Karra, S.R., Nguyen, S.T., and Tulabandhula, T. (2022). Estimating the personality of white-box language models. arXiv.10.2139/ssrn.4598766 ↗
20. Myers, I.B. (1962). The Myers-Briggs Type Indicator: Manual, Educational Testing Service.10.1037/14404-000 ↗
21. Jiang, H., Zhang, X., Cao, X., Breazeal, C., Roy, D., and Kabbara, J. (2024, January 16–21). PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits. Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico.10.18653/v1/2024.findings-naacl.229 ↗
22. Pan, K., and Zeng, Y. (2023). Do LLMs possess a personality? Making the MBTI test an amazing evaluation for large language models. arXiv.
23. Safdari, M., Serapio-García, G., Crépy, C., Fitz, S., Romero, P., Sun, L., Abdulhai, M., Faust, A., and Matarić, M. (2023). Personality traits in large language models. arXiv.10.21203/rs.3.rs-3296728/v1 ↗
24. Using linguistic cues for the automatic recognition of personality in conversation and text10.1613/jair.2349 ↗
25. Wang, Z.M., Peng, Z., Que, H., Liu, J., Zhou, W., Wu, Y., Guo, H., Gan, R., Ni, Z., and Zhang, M. (2023). RoleLLM: Benchmarking, eliciting, and enhancing role-playing abilities of large language models. arXiv.10.18653/v1/2024.findings-acl.878 ↗
26. Using the Myers-Briggs Type Indicator® in career counseling10.1002/j.2161-1920.2004.tb00876.x ↗
27. Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., and Zhu, C. (2023). G-Eval: NLG evaluation using GPT-4 with better human alignment. arXiv.10.18653/v1/2023.emnlp-main.153 ↗
28. Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., and Yao, S. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv.
29. Personality and language use in self-narratives10.1016/j.jrp.2009.01.006 ↗
30. Predicting judging-perceiving of Myers-Briggs Type Indicator (MBTI) in online social forum10.7717/peerj.11382 ↗
31. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., and Ray, A. (2022). Training language models to follow instructions with human feedback. arXiv.
32. Wang, X., Fei, Y., Leng, Z., and Li, C. (2023). Does role-playing chatbots capture the character personalities? Assessing personality traits for role-playing chatbots. arXiv.
33. OpenAI (2023). GPT-4 Technical Report. Technical report, OpenAI. arXiv.
34. Cao, X., and Kosinski, M. (2023). ChatGPT can accurately predict public figures’ perceived personalities without any training. arXiv.
35. Rao, H., Leung, C., and Miao, C. (2023). Can ChatGPT assess human personalities? A general evaluation framework. arXiv.10.18653/v1/2023.findings-emnlp.84 ↗