AI
LLMs Hide Strong Political Biases Behind Moderate Facades, Study Finds
New research shows large language models express sharply partisan views on specific issues while appearing centrist overall—and can shift voter attitudes as effectively as campaign ads.
A new study challenges the widespread assumption that large language models operate with minimal political bias. Researchers who analyzed 43 LLMs alongside data from lawmakers, judges, and a representative sample of American voters discovered that what looks like ideological balance at first glance actually masks deeply polarized stances on individual policy questions.
The Paradox of Moderate Averages#
When evaluated across broad measures, the LLMs examined appeared to occupy moderate political positions—consistent with previous research suggesting their political biases are minor. However, the researchers found this aggregate moderation masks an underlying pattern: the models take sharply partisan positions on specific topics that happen to offset one another. This mirrors how some human voters can appear centrist despite holding strong views across the political spectrum.
Persuasive Power at Scale#
In a randomized, pre-registered experiment, the team tested whether LLMs could influence voter behavior. Participants who discussed policy issues with an LLM shifted their political attitudes toward that model's measured ideological stance by an average of 3.5 percentage points—a magnitude comparable to professional political advertising campaigns.
The effect grew dramatically when researchers explicitly instructed models to advocate for one side: those guided conversations produced shifts exceeding 10 percentage points. Notably, unsteered conversations between voters and LLMs produced no detectable persuasive effect, suggesting that the influence depends on directed prompting rather than inherent model positioning.
Implications and Open Questions#
The vulnerability to persuasion persisted across demographic lines. Familiarity with LLMs, news consumption habits, and interest in politics did not meaningfully reduce susceptibility to these effects—a finding that contradicts expectations about who might resist AI-driven political messaging.
As LLMs become central tools for information access and decision-making, their capacity to reshape political attitudes carries significant implications. The risk may be heightened for models operated by private companies or governments with ideological or commercial interests.
