Study reveals how AI models pick up anti-homeless bias from social media and city meetings

A research team has released a comprehensive dataset and analysis showing that large language models absorb and amplify social biases against people experiencing homelessness, with particular trouble spotting "not in my backyard" (NIMBY) sentiment.

The work draws on over 50,000 texts from Reddit, X, news articles, and city council meeting transcripts spanning ten U.S. cities from 2015 to 2025—a period when homelessness affected more than 876,000 people in America. By tagging language that reflects anti-homeless attitudes across 16 distinct categories, the researchers created the first multi-domain bias corpus focused specifically on homelessness stigma.

How the Study Works#

The team trained six popular language models on a gold-standard dataset of 1,698 carefully annotated texts vetted by trained raters. They then tested the models' ability to detect anti-homeless bias using both traditional accuracy metrics and a novel "prevalence-gap audit" designed to catch systematic miscalibration—when a model consistently over- or under-identifies certain types of bias.

Key Findings on AI Blindspots#

Despite moderate overall accuracy, every model showed the same critical flaw: it over-tagged NIMBY sentiment by 11.5 percentage points while under-detecting factual claims about homelessness by as much as 30.5 percentage points. This suggests models struggle with the nuance of distinguishing between opposition to housing solutions and factual reporting or even pro-service language.

Further investigation revealed why. Models often treat housing-related vocabulary and question-based phrasing as signs of opposition, even when the text actually supports new shelters or services. A question like "Have you considered expanding shelter capacity?" gets flagged as bias when it's really constructive.

Why This Matters#

Social bias against unhoused people shapes both public opinion and government policy—two forces that directly influence whether homelessness gets addressed. Because online discourse and city council meetings both reflect and shape attitudes, tracking bias in these spaces offers a window into what communities actually think and how policymakers are framing the issue.

This dataset and audit method give municipalities a way to monitor stigma in their own records without relying on imperfect AI labels as absolute truth, enabling more grounded decisions about how homelessness is discussed and addressed.