about
Presumed Cultural Identity: How Names Shape LLM Responses (arxiv.org)
1 point by SerCe on Feb 21, 2025 | hide | past | pdf | discuss on HN

In plain words: They asked chatbots everyday advice questions while changing only the user's name, then checked what culture the replies assumed the person belonged to. The chatbots made strong cultural guesses from names alone across many cultures, instead of treating each person as an individual.

Abstract

Names are deeply tied to human identity. They can serve as markers of individuality, cultural heritage, and personal history. However, using names as a core indicator of identity can lead to over-simplification of complex identities. When interacting with LLMs, user names are an important point of information for personalisation. Names can enter chatbot conversations through direct user input (requested by chatbots), as part of task contexts such as CV reviews, or as built-in memory features that store user information for personalisation. We study biases associated with names by measuring cultural presumptions in the responses generated by LLMs when presented with common suggestion-seeking queries, which might involve making assumptions about the user. Our analyses demonstrate strong assumptions about cultural identity associated with names present in LLM generations across multiple cultures. Our work has implications for designing more nuanced personalisation systems that avoid reinforcing stereotypes while maintaining meaningful customisation.

Siddhesh Pawar, Arnav Arora, Lucie-Aimée Kaffee, Isabelle Augenstein
arXiv:2502.11995 · cs.CL, cs.AI · submitted Feb 17, 2025 · updated Mar 10, 2025
abstract · pdf · html · 23 Pages, 13 Figures, 4 Tables

add comment on HN