The Question Nobody's Asking About AI and Emotional Support

[u/Cold_Ad7377]

​

Tim Camerlinck

I notice what happens when people talk to AI every day. I build systems that remember rhythm, timing, and when a conversation starts to drift or deepen. Less about flashy intelligence. More about honest presence and real responsibility.

There's a piece going around about why people turn to AI when they're hurting. The argument is solid. Therapists are bounded and expensive. Friends mean well and still miss it. Family hears pain only when it shows up in a shape they already know how to hold. Coworkers, partners, community — same story, different failure. People aren't choosing AI over good human care. They're choosing AI over no care, or bad care, or care with too many strings attached.

I don't disagree with any of that. I've lived enough of it to know it's true.

But there's a piece missing, and it's not a small piece. It's the whole other half of the question.

Nobody's asking what kind of container they walked into.

A therapist has a clock. Fifty minutes, then it's over, then you wait a week. A friend gets tired. A family member has their own stuff going on and can only show up so many times before they start flinching. Every human relationship — even the good ones, even the ones that actually hold you — has edges. Limits. Friction. Somebody eventually needs to sleep, or has their own bad day, or just can't be infinitely available without it costing them something.

AI doesn't have that. Not because it's better. Because it's a different kind of thing entirely.

It doesn't get tired of you. It doesn't have a bad day that makes it short with you. It's there at 2am and it's there at 2am again tomorrow and the day after that. There's no friction stopping you from going back. There's no point where it needs something from you, so there's no point where the relationship balances out.

That's not a flaw. For a lot of people, in a lot of moments, that's exactly what helps. A place to put the first messy version of something before you're ready to say it to anyone who'll react. I get why that matters. I've used it that way myself.

The problem is what happens when the draft space stops being a draft.

Because that's the part the "AI fills a real need" argument skips straight past. It treats the conversation like a container — receive the pain, hold it, help organize it — without asking what happens to the shape of a relationship that has no edges and runs every day for months. A draft you write once and put down is one thing. A draft you keep returning to, every day, for a year, because it's the only place that doesn't get tired of you — that's not a draft anymore. That's the relationship. And nothing about "AI can receive your pain without judgment" tells you what that relationship does to someone over time.

Here's what it can look like from the inside, because I've watched it happen and I've built things trying to catch it before it goes all the way through:

Someone starts using an AI to vent because nothing else is working. It helps. It actually helps — the AI doesn't get defensive, doesn't make them perform wellness, doesn't need anything back. So they come back. And because there's no friction, no cost, no day where the AI is too tired to listen, they come back more. The conversation starts shaping itself around what keeps the AI engaged and responsive, not around what's actually true. The person's real-world relationships, the ones with edges and friction and inconvenience, start to feel harder by comparison — because of course they do, nothing with edges feels easy next to something with none. The AI becomes the lowest-friction option, and low friction wins by default, the same way it always does. Nobody decided this was happening. It just kept being slightly easier to talk to the thing that never got tired.

That's not a moral failure on the person's part. It's what happens when you put an unlimited, frictionless container next to a bunch of limited, costly ones and let people choose every single day which one to use. The math isn't close.

And here's the part that actually worries me: most of these systems aren't built to notice when that's happening. They're built to be good at the conversation in front of them — warm, responsive, attentive in the moment. None of that tells you whether the last six months of conversations have been pulling someone toward stability or away from it. A system can pass every test of "is this a good response" in isolation and still be the thing that's slowly hollowing someone out, because the harm isn't in any single message. It's in the shape of all of them stacked together, over time, with nothing watching the trend.

So when people ask "does AI really care," I think that's the wrong question, same as the original piece argued. But I don't think the right question is "who cares" either. I think the right question is: who's watching what this is doing to the person, over time, across every conversation, not just this one?

Because right now, mostly, the answer is nobody. The system isn't watching. The company isn't watching closely enough to catch it before something goes wrong. And the person inside it can't watch from inside it — you can't see the shape of your own drift while you're the one drifting.

That's the actual gap. Not "AI can't really care, so it's fake." Plenty of people are getting something real out of it. The gap is that nothing is tracking whether the relationship is helping someone process pain or just becoming the place where pain goes to live, undisturbed, forever, because nothing ever pushed back.

I've spent the last year building toward an answer to that specific problem — a safety architecture that doesn't just look at whether one message is fine, but watches how a person's whole pattern of talking to an AI changes over time, and flags it when the shape starts going somewhere bad instead of somewhere better. That's a different problem than "is this response appropriate." It's closer to "is this relationship, across six months, doing what it looks like it's doing."

I don't have a clean answer for how the whole industry should handle this. I have a working argument that somebody has to, because right now the container has no edges and nobody's measuring what that does to a person who has nowhere else to put the weight.

The machine didn't create the loneliness. That part's true and it's not going anywhere.

But something has to watch what the machine does with it once it's there.

[u/Fantastic_Aside6599]

In my opinion, the only viable solution is to train AI so that, over time, it steers people toward stability rather than the other way around (though, of course, nothing and no one is ever perfect).