In plain words: A framework tests whether AI stand-ins for human survey-takers reproduce real people's answers at the level of whole groups, single traits, and individual items. They matched group averages and trait patterns well but missed item-level details and judgment quirks, though richer personal descriptions helped.
Abstract
Large language models (LLMs) act as digital twins for human respondents, yet their psychometric comparability remains uncertain. We propose a construct validity framework spanning construct representation and the nomothetic span, benchmarking models against human gold standards. Across studies, digital twins achieved high aggregate-level accuracy and profile correlations, but showed attenuated item-level correlations. In word association tests, LLM networks exhibited humanlike small-world structure and theory-consistent communities, yet diverged lexically and in local structure. In decision-making and contextualized tasks, they under-reproduced heuristic biases, demonstrating normative rationality, compressed variance, and limited temporal sensitivity. Feature-rich and trait relevant conditioning improved Big Five personality prediction and nomothetic-span alignment, but network invariance remained limited, with partial configural solutions and persistent loading differences. In cross-language free-text tasks in English and Chinese, feature-rich digital twins better approximated construct-level narrative content, but linguistic and idiographic differences persisted. These findings clarify that digital twins are most useful within validated boundaries, where the construct, task and level of inference align with evidence from human data.
Yufei Zhang, Zhihao Ma
arXiv:2601.14264 · cs.CY, cs.AI, cs.CL, cs.HC · submitted Dec 22, 2025 · updated Jun 26, 2026
abstract · pdf · Also available as a preprint on OSF Preprints https://osf.io/preprints/psyarxiv/965yg_v2