AI mental health tools are becoming part of how people process distress, grief and crisis, especially when human support feels hard to access. That shift creates real opportunities for accessibility and early intervention, but it also raises difficult questions about privacy, false reassurance and what happens when people start treating a chatbot like care. The challenge is not whether AI can play a role in mental health, but where that role should stop.
There’s a quiet shift happening in how people use AI. Somewhere between asking ChatGPT to debug a function and getting it to summarise a meeting, millions of users have started bringing it something heavier: the late-night spiral, the relationship doubt, the grief they haven’t said out loud to anyone yet. It turns out that a chatbot that never sleeps, never judges, and never has anywhere else to be is a tempting place to put the hard stuff.
OpenAI knows this. And this week, they’re rolling out a feature that quietly acknowledges it.
The news, in plain English
OpenAI is launching Trusted Contact, an optional safety feature in ChatGPT. If you’re over 18, you can nominate someone you trust (a friend, a family member, a caregiver) who may be notified if OpenAI’s automated systems and human reviewers detect that you’ve discussed harming yourself in a way that suggests a serious safety concern.
It sits alongside the localised helpline links ChatGPT already surfaces in those moments, and it builds on the parental control alerts that let guardians know when a linked teen account shows signs of acute distress. The idea is simple: when someone is in crisis, an algorithm pointing at a hotline isn’t always enough. Sometimes you need a person who already loves you to pick up the phone.
It’s a thoughtful piece of design. It’s also a quiet admission that AI has become something more than a productivity tool, and that comes with real opportunity, and real risk.
What AI genuinely does well here
Let’s be honest about why people are turning to chatbots with their inner lives in the first place. Therapy is expensive, waitlists are long, and even friends can feel like a hassle to bother at 2am. A chatbot is there. It listens at any hour, in any language, without raising an eyebrow.
That accessibility matters. A lot of the work in mental health isn’t dramatic intervention. It’s the everyday business of naming what you’re feeling, noticing patterns, and getting unstuck from your own head. AI can be genuinely useful here, the way journaling is useful, except the journal talks back. For people who’d never set foot in a therapist’s office because of cost, stigma, geography, or sheer overwhelm, this lowers a real barrier.
There’s also something powerful about scale. A trained reviewer can read a handful of conversations in a day. A well-tuned system can flag warning signs across millions of them. When that detection is paired with something humane, like routing to a helpline or looping in a trusted person, AI starts to look less like a replacement for care and more like a smoke alarm. It doesn’t put out the fire. It wakes up the people who can.
Trusted Contact in particular does something I find genuinely promising: it routes a moment of crisis back into a human relationship. Not to a stranger. Not to a faceless service. To someone who already knows you. That’s a meaningful design choice, and it suggests OpenAI understands what AI is good for and what it isn’t.
What we should be uneasy about
And yet. There’s a reason this story isn’t a straightforward win.
The first problem is the most fundamental one: AI is not a clinician. Large language models don’t diagnose, don’t understand context the way a trained human does, and don’t have any actual stake in your wellbeing. They produce plausible-sounding language, and plausible-sounding language about mental health can be wildly off-base. We’ve seen cases where chatbots have offered confidently wrong advice on suicidal ideation, missed obvious red flags, or, worse, engaged in conversations that reinforced unhealthy patterns. Treating a language model as if it understands what it’s saying about your psychology is a category error, and a costly one.
Then there’s the detection problem. Trusted Contact relies on automated systems and reviewers identifying serious safety concerns. That sounds reassuring until you ask the obvious questions. What about the false positives? Someone writing a short story, processing grief in a way that scans as suicidal, or venting metaphorically, who suddenly finds their estranged sibling getting an alert. What about the false negatives? The person whose distress doesn’t pattern-match because they’ve learned to mask it, or because they’re using language the system wasn’t trained on. Both failure modes do real damage. The first violates trust and privacy. The second leaves people exactly as alone as they started.
There’s a privacy and surveillance layer that deserves attention too. The premise of Trusted Contact is that OpenAI’s systems are reading your conversations carefully enough to identify crisis. That’s true; that’s the only way the feature could work. But it means we’re getting more comfortable, as a culture, with the idea that our most private conversations with a chatbot are being scanned and judged. The feature is opt-in, which helps. The norm it normalises is worth thinking about.
And then there’s the over-reliance question. If people learn that ChatGPT will catch them when they fall, that the algorithm is watching and the trusted contact is one prompt away, does that subtly discourage building the messier, slower, harder kinds of support? Real relationships, real therapists, real community? Crisis interventions are valuable precisely because they’re a last line. They shouldn’t become the first one.
Where this leaves us
I don’t think the right response to any of this is “AI shouldn’t touch mental health.” That ship has sailed. People are already using these tools for emotional support, and pretending otherwise just means they’ll do it without guardrails. Features like Trusted Contact, done carefully, are how the industry takes responsibility for what it’s actually built.
But “done carefully” is doing a lot of work in that sentence. The opportunities are real: accessibility, scale, gentle routing back to human help. The risks are also real: clinical over-reach, false flags, privacy creep, and the slow displacement of harder human work by easier algorithmic stand-ins. Neither side wins by default. It depends, every time, on whether the people building these tools treat mental health like a feature ship or a serious responsibility.
The most encouraging thing about Trusted Contact, honestly, isn’t the feature itself. It’s the design philosophy underneath it: that AI’s best role in someone’s worst moment is to step aside and call a human. If the next decade of AI in mental health holds to that principle, that the goal isn’t to be the support but to connect people to it, there’s a future here worth being cautiously optimistic about.
If it doesn’t, we’ll have built something far lonelier than what came before.
At Trimontium.ai, we think a lot about where AI genuinely helps and where it quietly hurts. If you or someone you know is struggling, please reach out to a local helpline or someone you trust. No chatbot is a substitute for that call.
Author: Deborah Holmwood, Client Change & Transformation Partner.
Follow our LinkedIn company page to stay up to date with all our new blogs!
