AI Therapy Apps: Risks, Regulation & New Safety Guidelines

Millions are turning to digital tools for mental well-being, seeking support through apps that promise everything from guided meditation to AI-powered therapy. But are these applications best understood as modern equivalents of a doctor’s visit, a yoga class, a pharmaceutical intervention, or simply another form of self-care, like a dietary supplement? The lines are increasingly blurred, and a recent study from Cornell University highlights the urgent need for clearer guidelines around the development and deployment of these technologies.

The growing popularity of these mental health apps, many of which leverage large language models (LLMs) like ChatGPT, has outpaced regulatory oversight. This leaves users vulnerable to potentially ineffective or even harmful interventions. Researchers at Cornell are now proposing a framework for evaluating these apps, urging developers to consult with experts and align with existing state and federal regulations. Their perform, published earlier this month, suggests a need to categorize these tools based on the guarantees they offer and the reliability of those guarantees.

Defining the Spectrum of Digital Mental Health

The Cornell team proposes thinking about mental health apps along a spectrum. At one end are tools that make explicit claims about delivering specific benefits – akin to a drug promising to reduce anxiety. These apps would require rigorous testing and validation, similar to pharmaceutical trials. At the other end are apps offering general wellness support, like a yoga app, where benefits are less defined and more reliant on individual effort. The researchers suggest that the level of scrutiny should match the level of claim.

This isn’t a new debate. For decades, the efficacy of various mental health treatments has been a subject of intense study. Traditional therapies, like cognitive behavioral therapy (CBT), have a substantial evidence base, but even these approaches aren’t universally effective. The challenge with apps is the sheer diversity of approaches and the lack of standardized evaluation. A recent analysis, detailed in a LinkedIn post by Duncan Green, examined over 1,750 research papers that acknowledged using ChatGPT. The study revealed that 80% of researchers used the tool for text editing and proofreading, a far cry from its potential for complex therapeutic interventions. Witness the full analysis here.

The Rise of LLMs and the Editing Effect

The Cornell study builds on growing concerns about the impact of artificial intelligence on scientific research. A December 2025 report from the same institution highlighted a surge in “mediocre papers” attributed to the use of LLMs. Read more about the impact of AI on research quality. While these tools can undoubtedly assist with tasks like writing and editing, they don’t necessarily enhance the quality of the underlying research or provide genuine therapeutic benefit. The LinkedIn analysis further supports this, showing that only a small fraction of researchers (5.3%) credited ChatGPT with tasks beyond editorial assistance, such as coding or data analysis.

This “editing effect” is particularly relevant to mental health apps. Many apps use LLMs to generate personalized content or provide conversational support. Yet, the quality of this content depends heavily on the underlying algorithms and the data they are trained on. Without careful oversight, these apps could perpetuate biases, offer inaccurate information, or even exacerbate existing mental health conditions.

What Does This Mean for Users?

For individuals seeking mental health support through apps, the current landscape requires a degree of caution. It’s crucial to understand what the app is claiming to offer and whether those claims are supported by evidence. Look for apps that have been vetted by qualified mental health professionals and that adhere to established ethical guidelines. The Food and Drug Administration (FDA) has begun to explore regulation in this space, but progress is slow. Currently, the FDA primarily regulates apps that function as medical devices, such as those used to diagnose or treat specific conditions. Many mental wellness apps fall outside of this regulatory framework.

It’s also important to remember that mental health apps are not a substitute for professional care. If you are struggling with a mental health condition, it’s essential to consult with a qualified clinician. Apps can be a useful supplement to traditional therapy, but they should not be relied upon as the sole source of support. The National Institute of Mental Health (NIMH) offers resources for finding mental health professionals and accessing evidence-based treatments. Find resources at NIMH.

Navigating the Regulatory Landscape

The Cornell researchers emphasize the need for a multi-faceted approach to regulation. This includes consulting with experts in mental health, technology, and law, as well as reviewing existing state and federal regulations. They propose four distinct categories for LLM-based mental well-being apps, based on the guarantees they offer and the reliability of those guarantees. This framework could help regulators determine the appropriate level of scrutiny for each type of app.

The European Union is also actively considering regulations for AI, including those used in healthcare. The proposed AI Act aims to establish a risk-based framework for regulating AI systems, with stricter requirements for high-risk applications, such as those used in medical diagnosis or treatment. These international efforts could influence the development of regulations in the United States.

Looking Ahead: A Call for Responsible Innovation

The development of mental health apps powered by LLMs holds immense potential, but it also carries significant risks. The Cornell study serves as a crucial reminder that innovation must be guided by ethical considerations and a commitment to user safety. The researchers are advocating for a proactive approach to regulation, one that prioritizes the well-being of individuals seeking mental health support.

The next steps involve ongoing research to evaluate the effectiveness of these apps, the development of standardized evaluation metrics, and the implementation of clear regulatory guidelines. It also requires open dialogue between developers, regulators, and mental health professionals to ensure that these technologies are used responsibly, and ethically. The goal is to harness the power of AI to improve mental health outcomes, while safeguarding against potential harm.