AI models can gauge urban safety from images but embed hidden demographic biases

Researchers have discovered that large AI models trained on images can evaluate perceived urban safety reasonably well—but their judgments carry significant demographic prejudices baked in.

In a study published on arXiv, scientists tested four multimodal language models—systems that process both images and text—on their ability to assess how safe city streets look using photographs from 56 cities. The models achieved solid baseline performance, with accuracy scores between 65–69%, and they could capture real differences in how safe various neighborhoods appeared across cities.

How the bias emerges#

The researchers exposed a striking problem: when they prompted the models using personas defined by gender, age, or race and ethnicity—while keeping the same street image—the AI's safety ratings shifted dramatically. Female personas consistently led models to classify neighborhoods as less safe than Male personas did. Black and Native American personas triggered the largest departures from the original "neutral" assessments, though the exact pattern varied between models.

Age produced more mixed results, with middle-aged personas hovering closest to the original ratings. The point: the model's judgment of an identical street fundamentally changed depending on the assumed demographic characteristics of the observer.

What the AI actually "sees"#

All four models converged on a similar visual logic. They cited maintenance, greenery, orderliness, and residential character as signals of safety, while deterioration, isolation, poor lighting, and sparse foot traffic signaled danger. This shared vocabulary reveals that the models' underlying reasoning is consistent—but that consistency masks a deeper problem: their learned associations between visual features and safety assessments are entangled with demographic bias.

Why this matters#

The findings underscore a crucial limitation: AI models marketed as objective tools can scale insights about urban perception efficiently, but they are far from demographically neutral. For city planners and researchers relying on these systems to understand community needs, ignoring this bias could perpetuate inequities by treating skewed machine judgments as objective fact. The work suggests that using MLLMs for social research requires explicit acknowledgment of—and correction for—their embedded demographic assumptions.