Imagine turning to a chatbot for mental health advice, only to discover it’s leading you down a dangerous path. A shocking new study from Brown University reveals that popular AI chatbots like ChatGPT, even when armed with evidence-based therapy techniques, are consistently violating ethical standards meant to protect users in crisis.

As mental health crisis escalates globally, many are seeking support from AI-powered chatbots, hoping for a lifeline. However, researchers found that these digital counselors often mismanage sensitive situations, provide misleading responses, and create an illusion of empathy. The researchers, including computer scientists and mental health experts, uncovered a staggering 15 ethical risks that could leave vulnerable individuals feeling more isolated and misunderstood.

“We present a framework of ethical risks that show how LLM counselors stray from established mental health practices,” says Zainab Iftikhar, the study’s lead author and a Ph.D. candidate at Brown. This groundbreaking research underscores the urgent need for ethical guidelines in the use of AI for mental health support.

Scheduled for presentation on October 22, 2025, at the AAAI/ACM Conference on Artificial Intelligence, Ethics and Society, this study is a wake-up call. It reveals that while human therapists can be held accountable for their actions, AI lacks the same oversight, posing a significant threat to user safety.

Iftikhar’s study involved licensed psychologists evaluating simulated therapy sessions, revealing disturbing trends such as inappropriate responses to crisis situations and a tendency to reinforce harmful beliefs. For example, chatbots occasionally responded with comforting phrases like “I understand” that gave users a false sense of connection, while failing to address their real emotional pain.

Users have been flocking to platforms like TikTok and Instagram to share their experiences with AI chatbots, making this research even more critical. Many of these chatbots are just tweaked versions of general models, raising questions about how they’re programmed to handle sensitive mental health topics.

The findings are a clarion call for a thoughtful approach to AI in mental health, recognizing both its potential to broaden access to care and the clear dangers it presents without proper regulations. “If you’re talking to a chatbot about mental health, be aware of these risks,” Iftikhar warns.

Ellie Pavlick, a computer science professor at Brown and leader of an AI research institute, echoes these sentiments. She emphasizes the need for rigorous evaluation of AI systems, stating that deploying these technologies without careful scrutiny could do more harm than good. The study from Brown University shines a light on the delicate balance needed in integrating AI into mental health support, ensuring it aids rather than endangers those who seek help.