New benchmark reveals how AI models struggle with accurate harm reduction guidance for drug users

A new research benchmark has exposed critical gaps in how large language models handle harm reduction information—evidence-based, non-judgmental guidance designed to help people who use drugs protect their health and safety.

The HarmReduction benchmark#

Researchers have developed the HarmReduction benchmark to systematically test how well contemporary LLMs perform on tasks central to substance use safety. The evaluation framework includes HR-Basic, a dataset containing 2,160 question-answer-evidence pairs spanning three core competencies: verifying safety boundaries, providing accurate numerical data about dosing or drug effects, and assessing risks when multiple substances are combined.

The benchmark assesses models in two distinct operating modes: one that relies solely on their trained knowledge, and another that integrates external domain expertise through retrieval-augmented generation (RAG)—a technique that allows models to draw on specialized reference materials.

Finding gaps in model performance#

The results paint a concerning picture. Even state-of-the-art LLMs demonstrate substantial weaknesses when asked to deliver harm reduction information. Beyond accuracy challenges, the evaluation uncovered instances where models provided guidance that could pose serious health risks to users seeking support.

This inconsistency undermines a key promise of AI in public health: that sophisticated language models could help meet the information needs of vulnerable populations who might face stigma or barriers to accessing traditional healthcare services.

Implications for public health#

The work underscores a broader tension in deploying LLMs in sensitive health contexts. While models have shown strong reasoning abilities across medical domains, their application to substance use—an area requiring both technical precision and nuanced, harm-centered thinking—remains unreliable. The findings suggest that without careful validation and targeted improvements, relying on LLMs for harm reduction information could inadvertently harm rather than help.

The research contributes an important evaluation framework to help guide future development and deployment of AI tools in harm reduction and related public health applications.