Names Alone Shift LLM Token Predictions in Patterns Tied to Race and Gender

New research reveals that large language models shift their internal probability distributions when presented with names associated with different racial and gender groups—even before generating a single token. These statistical shifts occur automatically, signaling hidden demographic sensitivities baked into how modern AI systems process language.

What the Study Found#

Researchers tested six families of open-weight language models using 5,760 sentence-completion prompts where only a first name indicated the subject's race and gender. They measured Shannon entropy—a standard metric for prediction uncertainty—across the full vocabulary of possible next tokens.

The patterns were striking: names associated with Black individuals consistently triggered higher entropy and more varied continuations compared to names associated with White individuals. This held true across all six instruction-tuned models and all six base checkpoints, suggesting the bias is deeply embedded rather than accidental. The disparity reversed for gender, with effects additive to race.

However, the size of individual effects was modest (Cohen's d = 0.06–0.16), though when aggregated across prompted templates, the template-level differences grew substantially (d = 0.66–1.08).

A Surprising Reversal#

The findings directly contradict prior work by Lee et al. (2024) on explicit group-label probing, which had documented the opposite pattern—what researchers termed a "homogeneity bias" favoring uniform responses about specified demographic groups. The key difference appears to lie in how the demographic signal is conveyed: implicitly through a name versus explicitly stated as a label.

When researchers shifted to chat-formatted prompts, the entropy disparities shrank noticeably. Direct probing with explicit group labels yielded mostly null or reversed results.

How Methodology Shapes Findings#

A critical insight emerged: the choice of experimental method does not merely measure bias—it fundamentally shapes which disparities become visible and in which direction. The researchers pinpointed the source of output diversity differences not in uniform statistical skew but in heterogeneity across name-conditioned continuations—a dimension that a fixed group label cannot capture.

This suggests that previous bias studies relying on explicit demographic statements may have missed important variation patterns that emerge only through more implicit signaling mechanisms like names.