New tool helps people critically examine their AI therapy conversations

As AI tools increasingly take on roles in mental health support, a research team has developed a novel approach to make these interactions safer: helping people think more carefully about their own conversations with the technology.

The reflection tool#

Researchers introduced CounselReflect, a framework that takes established counseling quality standards from the literature and transforms them into practical questions users can ask themselves about their AI mental health conversations. The tool serves as a guide for examining the depth, appropriateness, and effectiveness of advice or support received.

What the research found#

The team interviewed 21 people who regularly use AI for mental health and well-being discussions. While most said they didn't typically pause to evaluate these conversations, nearly all identified specific dimensions they would want a reflection tool to address. However, the study revealed a troubling pattern: when participants actually used the tool to review their interactions, they tended to seek out evidence that confirmed what they already believed about the AI—rather than exposing gaps in their understanding.

The key challenge#

A major obstacle emerged: people showed selective attention, focusing reflection on qualities they already valued while overlooking potentially important blind spots in how they engaged with the system. Additionally, the emotional weight of revisiting difficult or tense conversations proved to be a significant barrier that participants struggled to overcome. The researchers emphasize that reflection tools need to actively surface overlooked issues and encourage more comprehensive examination of AI interactions, rather than simply presenting passive metrics.

The findings underscore the complexity of designing genuinely useful reflection mechanisms at a time when AI adoption in mental health care is outpacing the development of safety standards and governance frameworks.