Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Prompts

Summary

This paper presents a multi-dimensional analysis of human-like behaviors in large language models (LLMs), focusing on their prevalence, potential effects, and controllability. Using LLM-as-a-judge and human evaluation, the authors analyzed 21,000 multi-turn conversations from four widely used models: gpt-4o, gpt-4.1-mini, claude-sonnet-4.6, and gemini-2.5-flash. They found that human-like behaviors are pervasive but vary across models and user factors such as conversation goals and user profiles. Human evaluators judged self-referential and relationship-building behaviors as less appropriate from LLMs than from humans, but boundary-maintaining behaviors as more appropriate from LLMs. The study also demonstrates that system prompting can control these behaviors, though careful evaluation is required to avoid unintended effects. The findings have implications for responsible LLM design and evaluation.