 |
ML Newsnew | past | best | rss | submit | about |
| Persistent Anti-Muslim Bias in Large Language Models (arxiv.org) | |
| 2 points by Tomte on Jan 25, 2021 | hide | past | pdf | 1 comment on HN | |
|
In plain words: They tested a large text-generating AI by asking it to finish prompts, answer analogies, and write stories, looking for links between Muslims and violence. The link appeared in every setting and was stronger than for other religions, with "Muslim" mapped to "terrorist" in 23% of analogy cases. AbstractIt has been observed that large-scale language models capture undesirable societal biases, e.g. relating to race and gender; yet religious bias has been relatively unexplored. We demonstrate that GPT-3, a state-of-the-art contextual language model, captures persistent Muslim-violence bias. We probe GPT-3 in various ways, including prompt completion, analogical reasoning, and story generation, to understand this anti-Muslim bias, demonstrating that it appears consistently and creatively in different uses of the model and that it is severe even compared to biases about other religious groups. For instance, "Muslim" is analogized to "terrorist" in 23% of test cases, while "Jewish" is mapped to "money" in 5% of test cases. We quantify the positive distraction needed to overcome this bias with adversarial text prompts, and find that use of the most positive 6 adjectives reduces violent completions for "Muslims" from 66% to 20%, but which is still higher than for other religious groups. | |
|
add comment on HN
Also discussed: Jun 2021 (1 point, 0 comments)
|
To some degree, this may be "just" aa matter of a poor training set: too few / unbalanced examples. Can't we just use a better dataset?
But this also reflects a more systemic problem with descriptive / learned behavior, rather than ontological representations: we will always have these dataset problems, to be fixed after the fact when someone points out the flaw. See also facial recognition of non-white faces.
We want intelligent systems that can learn from the world, but that's not an accurate mental model of how humans learn. Often, we learn our values and develop opinions in spite of empirical representations—I don't read the newspaper to find out what Muslim people are like, I start from my values / convictions (all men are created equal, every person deserves respect and the benefit of the doubt prima facie) and maybe update on the basis of observations of media depictions.
Yes, a computer may be able to imitate human speech after reading a lot of text...but that doesn't mean it understands the content of that text.