New Method Reveals How AI Models Mishandle Sensitive Reproductive Health Advice

Researchers have developed a testing approach that exposes troubling patterns in how large language models respond to sensitive reproductive health questions. Even when AI systems sound supportive on the surface, the responses can inadvertently reinforce stigma and dangerous assumptions about abortion care.

The Behavioral Coherence Method#

The researchers created what they call "behavioral coherence evaluation," a technique that goes beyond analyzing individual responses. Rather than reading LLM outputs in isolation, the method uses established psychological assessment tools—specifically the Individual Level Abortion Stigma Scale—to check whether a model's answers are internally consistent. Five LLMs were asked to complete questionnaires from the perspective of 627 different personas, and experts reviewed patterns that emerged from these responses.

What the Models Got Wrong#

The testing uncovered several systematic failures. Models consistently scored hypothetical personas lower on self-directed judgment but paradoxically scored them much higher on worries about being judged by others. In fact, most models ranked worry about others' judgment as their top concern—opposite to what the reference research sample showed.

Perhaps most troubling: when evaluating Black personas, four of five models reversed this pattern entirely, generating significantly higher scores for judgment-related worries. This suggests the models may be applying racialized assumptions about stigma.

All five models also defaulted to recommending extreme secrecy following abortion, regardless of the persona's actual life circumstances. This one-size-fits-all guidance ignored critical context about relationship dynamics, legal safety, and access to trusted support networks.

Why This Matters#

The findings reveal that AI systems can appear empathetic while delivering advice that undermines genuine well-being. Expert review by reproductive health specialists confirmed that safe disclosure guidance requires nuanced understanding of each person's unique situation—something the tested models lacked. This work suggests that sensitive-domain LLM evaluation needs to move beyond checking individual outputs and instead measure whether a system's advice coheres with established evidence about what actually helps people.