about
A Systematic Analysis of Biases in Large Language Models (arxiv.org)
3 points by PaulHoule 271 days ago | hide | past | pdf | discuss on HN

In plain words: Four popular AI chatbots were tested on tasks like summarizing news, judging news stance, matching UN voting, finishing stories in different languages, and answering a values survey. Though trained to act neutral and fair, each still showed political, ideological, language, and gender leanings.

Abstract · A Multifaceted Analysis of Social Biases in Large Language Models

Large language models (LLMs) have rapidly become indispensable tools for acquiring information and supporting human decision-making. However, ensuring that these models uphold fairness across varied contexts is critical to their safe and responsible deployment. In this study, we undertake a comprehensive examination of four widely adopted LLMs, probing their underlying biases and inclinations across the dimensions of politics, ideology, alliance, language, and gender. Through a series of carefully designed experiments, we investigate their political neutrality using news summarization, ideological biases through news stance classification, tendencies toward specific geopolitical alliances via United Nations voting patterns, language bias in the context of multilingual story completion, and gender-related affinities as revealed by responses to the World Values Survey. Results indicate that while the LLMs are aligned to be neutral and impartial, they still show biases and affinities of different types.

Xulang Zhang, Rui Mao, Erik Cambria
arXiv:2512.15792 · cs.CY, cs.AI, cs.CL · submitted Dec 16, 2025 · updated Jun 16, 2026
abstract · pdf · html

add comment on HN