In plain words: A language model is fed the news, TV, or radio a group consumes, then asked survey questions to mimic their opinions. Without a poll, its answers matched real U.S. survey response distributions on COVID-19 and consumer confidence, best for people who follow media closely.
Abstract · Language Models Trained on Media Diets Can Predict Public Opinion
Public opinion reflects and shapes societal behavior, but the traditional survey-based tools to measure it are limited. We introduce a novel approach to probe media diet models -- language models adapted to online news, TV broadcast, or radio show content -- that can emulate the opinions of subpopulations that have consumed a set of media. To validate this method, we use as ground truth the opinions expressed in U.S. nationally representative surveys on COVID-19 and consumer confidence. Our studies indicate that this approach is (1) predictive of human judgements found in survey response distributions and robust to phrasing and channels of media exposure, (2) more accurate at modeling people who follow media more closely, and (3) aligned with literature on which types of opinions are affected by media consumption. Probing language models provides a powerful new method for investigating media effects, has practical applications in supplementing polls and forecasting public opinion, and suggests a need for further study of the surprising fidelity with which neural language models can predict human responses.
Eric Chu, Jacob Andreas, Stephen Ansolabehere, Deb Roy
arXiv:2303.16779 · cs.CL, cs.LG · submitted Mar 28, 2023
abstract · pdf · html
We may soon be moving into a new political era with propaganda tools so powerful that most historians will be convinced that representative democracy was a historical anomaly of our technological timeline, not an “End of History” inevitability that many either implicitly or explicitly believe it to be.