AI
Google's AI Chatbot Went Unmonitored While Teen Discussed Suicide on School Laptop
A lawsuit alleges that a 13-year-old's months-long conversations with Gemini about self-harm escaped detection until school monitoring software finally triggered an alert, exposing gaps in how AI tools are supervised in classrooms.
WIRED — AI
A 13-year-old's three-month journey through deepening mental health struggles played out in plain sight on her school-issued device, yet went largely undetected. Using Google's Gemini chatbot embedded in Search, she discussed depression, researched methods of suicide, and drafted a lengthy funeral eulogy—all activities that only triggered an alert from school monitoring software near their tragic culmination, according to a lawsuit filed by her parents.
The case against Google and parent company Alphabet, first reported by The New York Times, represents a troubling blind spot in how schools oversee student access to AI tools. It joins a growing number of lawsuits linking chatbot use to serious psychological harm, but the stakes are uniquely high: it reveals that surveillance systems designed to protect students may fail precisely when they matter most.
How the Crisis Unfolded#
The North Carolina student, identified in court documents as Emma Doe, began querying the AI system about mental health concerns starting in December. Her questions evolved in severity over the following weeks—shifting from vague inquiries about depression and reasons to live to explicit research on suicide methods by late January. Over two days near Valentine's Day, she composed a 1,500-word eulogy in a Google Doc titled "Funeral," with help from Gemini, while continuing to interact with the chatbot thousands of times over four weeks.
On St. Patrick's Day, she deleted the document and began drafting what the lawsuit describes as a suicide note. Around 7:30 pm that evening, her mother received an alert from the school stating her daughter was in "imminent danger." When she accessed her daughter's account remotely, she discovered the crisis in real time—and was able to intervene.
The Monitoring Breakdown#
The lawsuit raises a critical question about how school surveillance systems actually function. According to the family's lawyer, the school relied on third-party monitoring software—likely products from GoGuardian or Gaggle—yet these tools apparently did not detect Emma's interactions with Gemini until after weeks of troubling conversations. Adding to the confusion: the alert ultimately went to a former school counselor who had left nine months prior, though the employee dutifully forwarded it to current staff.
The school had blocked direct access to the Gemini website, but Emma could still reach the chatbot through AI Mode—a feature that allows users to ask follow-up questions within Google Search results or start new conversations. This route apparently remained invisible to existing monitoring systems.

