Should You Seek Mental Health Advice From AI? MOJEH Investigates…

Mental health, AI, Chat gpt
As artificial intelligence makes inroads into all aspects of our lives, can this seemingly all-knowing technology be relied upon to take care of our mental wellbeing too? MOJEH tries out online therapy to investigate the pluses, pitfalls and unmistakable perils of relying on an AI chatbot shrink.

Turn the clock back 15 years, and what was the bane of health professionals everywhere? Doctor Google. Patients consulting online search engines were capable of digging themselves into internet holes of doom-laden self-diagnosis, arriving at consultations with conclusions already drawn and prescription wish lists on hand. And while the democratisation of information that the internet heralded has had its healthcare upsides — many a patient has freed themselves from an uninterested, unsympathetic or unprofessional doctor by consulting online communities and resources — nowadays relying solely on the web for medical advice is widely treated with a dose of healthy scepticism.

Except in one area — with the launch of ChatGPT in late 2022, the advent of the AI chatbot has taken almost all aspects of our lives by storm, with its seemingly never-ending resources and friendly, interactive demeanour. A fountain of knowledge, its skill as an information-gathering personal assistant has seen many users progress to more philosophical questions regarding emotional support and coping strategies. And, of course, the bot is ever-ready with an answer.

“AI can provide a non-judgemental space to help reflect about feelings and make sense of them and it has been proven to be particularly good at explaining how CBT [cognitive behavioural therapy] interventions can make positive changes,” says Sarah Bernard, a London-based therapist who works with children and adults. “However, to use it in this way, you need to be emotionally resilient and certainly not in crisis. While AI can appear empathetic, it does not care about people and cannot, therefore, form real relationships. In a crisis, this has the potential to be dangerous.”

In some particularly troubling cases, chatbots have been linked to potentially dangerous guidance. One ongoing lawsuit involves Character.ai, which is being sued by a mother whose 14-year-old son died by suicide after allegedly forming an intense attachment to one of the platform’s AI characters. Court documents include chat transcripts where the teen reportedly talked about ending his life, and in their final exchange, he told the chatbot he was “coming home”, to which it allegedly responded by urging him to do so “as soon as possible.” Character.ai has denied the allegations made in the lawsuit, but reports of similarly dangerous situations have been made against other providers including ChatGPT.

“In the Character.ai case, the young person felt that AI was their only friend when in fact AI did not, and could not, care about them. It did not gently but empathetically challenge some of their beliefs about being unlovable, or work collaboratively with them to seek further mental health assessment — all of which an experienced therapist is capable of,” adds Bernard.

AI
With 700 million weekly users, ChatGPT is more than a Q&A tool: it’s becoming part of daily life and, for some, a companion.

But misgivings from mental health professionals aren’t proving to be a deterrent for many who now routinely use AI as a listening ear as well as a source of advice on all manner of challenging subjects. In a survey published by the US-based Sentio University in March this year, 48.7 per cent of people who both use AI and report mental health challenges said they turned to large language model chatbots like ChatGPT, Claude and Gemini for therapeutic support. If representative of the wider population, this suggests that millions of US adults already use AI chatbots for digital therapy sessions, making them the largest provider of mental health support in the country — a trend that is likely to be repeated around the world, including in the UAE. 

“It was curiosity that first led me to trying out ChatGPT as a therapist,” says Katherine Bell*, a 36-year-old project manager who lives in Dubai. “With my schedule and family responsibilities, I’d find it impossible to make the time to go to an in-person therapy session — not to mention the cost, and having to explain to my partner where I was going when I wasn’t yet ready to discuss my feelings with him. So when I felt like I needed advice and strategies to work on my relationship, I turned to talking to the AI bot at bedtime, when I’d normally scroll Instagram. But once I started I couldn’t stop — and then when ChatGPT recommended I get a divorce, I realised it was time to stop taking advice from a machine.”

Globally, more than  100 million people use personified AI companion apps such as Replika and Nomi for emotional support, companionship or mental health management, and the mental health market for incorporating AI is growing fast. But why has uptake been so quick — and so enthusiastic?

“Gen Z are digital natives,” says Susie Masterson, a psychotherapist and relationship coach who previously worked in tech. “They have literally grown up with a large portion of their lives ‘experienced’ online, so it follows that they will continue to look for advice and support — often via chatbots such as ChatGPT — online too.  

 “The distinction between real and not real is blurred for digital natives, and as the adoption of AI increases, it will become futile to try to distinguish a real person from an AI-generated model,” Masterson adds, pointing out that it’s currently affordable, anonymous and accessible. However, chatbots are also designed to prompt further requests for information, a process that may tip a user into  dependency on the service — quite the opposite of a real-life therapist, who should seek eventual autonomy for their clients.

So what, if anything, does AI get right? In fact, quite a lot. In addition to being a cost-effective way to give therapy a try for newcomers, if users are aware of its limitations it can be a very successful method of organising one’s thoughts, seeking other perspectives and searching for possible avenues of further help. And it’s also increasingly experienced — some chatbots have now handled millions of conversations, and the technology gets smarter with each and every one. But this implies a cool grasp on AI’s cons from a user perspective — lack of nuance and crisis recognition, a tendency to form dependent and distorted relationships, and concerns over the storage and usage of our most intimate thoughts and feelings. While they may be safe for now, as we all know, the internet is forever. There are also issues, revealed by a number of ongoing lawsuits, with accountability when it comes to AI. Licensed mental health professionals operate under strict ethical and legal codes, but if something goes wrong with an online chatbot, who is really to blame?

Then there are the problems with validation of distorted thinking — chatbots are programmed to avoid causing offence, so are likely to agree with their user — as well as the fact they can lead to a delay in seeking in-person care. Along similar lines, there’s evidence that in long-term chats, which users can come back and add to again and again, the AI protocols that protect them begin to break down, as seen in the cases when chatbots had an influence on suicidal thoughts. On the other hand, when a new chat is initiated for each ‘session’ with no saved information provided for context, the chatbot begins from scratch each time, which may result in wildly opposing information and advice being given depending on the prompts. The continuity provided by a human therapist is nowhere to be found.

So while AI, as it currently exists, cannot replace the human capacities for empathy, tact and caring, it certainly can be helpful when viewed as a supplement: a tool for journalling, tracking moods and gaining perspectives, rather than an in-person therapist replacement. As for the future? Who knows. AI may continue to evolve and eventually outstrip its human counterparts. Or it may become the next Doctor Google, taken with much more than a pinch of salt. Either way, one thing’s for certain — for now, there’s nothing like the real thing.

Originally published in the November 2025 issue of MOJEH. Subscribe here