In plain words: A test checks whether AI chatbots' stated values match the actions they choose in realistic situations across twelve cultures and eleven social topics. The match was poor and varied widely, so stated values alone are a shaky guide to what these systems will do.
Abstract
Existing research primarily evaluates the values of LLMs by examining their stated inclinations towards specific values. However, the "Value-Action Gap," a phenomenon rooted in environmental and social psychology, reveals discrepancies between individuals' stated values and their actions in real-world contexts. To what extent do LLMs exhibit a similar gap between their stated values and their actions informed by those values? This study introduces ValueActionLens, an evaluation framework to assess the alignment between LLMs' stated values and their value-informed actions. The framework encompasses the generation of a dataset comprising 14.8k value-informed actions across twelve cultures and eleven social topics, and two tasks to evaluate how well LLMs' stated value inclinations and value-informed actions align across three different alignment measures. Extensive experiments reveal that the alignment between LLMs' stated values and actions is sub-optimal, varying significantly across scenarios and models. Analysis of misaligned results identifies potential harms from certain value-action gaps. To predict the value-action gaps, we also uncover that leveraging reasoned explanations improves performance. These findings underscore the risks of relying solely on the LLMs' stated values to predict their behaviors and emphasize the importance of context-aware evaluations of LLM values and value-action gaps.
Hua Shen, Nicholas Clark, Tanushree Mitra
arXiv:2501.15463 · cs.HC, cs.AI, cs.CL · submitted Jan 26, 2025 · updated Sep 30, 2025
abstract · pdf · html · EMNLP 2025 Main Paper
Instead you need to present scenarios which put these beliefs into effect. This is similar to how humans may claim to value sustainability but that doesn't mean when making a purchasing decision they will always opt for an eco-friendly option.