and because chatbots are designed to be agreeable and mirror the user’s ideas, they lack the complexity and reality-checks that human relationships provide.
a relationship with a chatbot does not require the kind of vulnerability and commitment that one must learn to bring into a human relationship. in a paper published last month, mit researcher cathy fang and her team highlight that brief interactions with chatbots can reduce loneliness — but heavier use
ultimately increases it.
the risk is that lonely teens further isolate themselves as they replace human relationships with artificial ones.
using chatbots to deal with mental health issues, then, poses serious risks. their depth of understanding and ethical safeguards simply do not meet the standards of professional care.
another study, published in the
journal of medical internet research, found that up to one-third of interactions with chatbots could lead to harmful suggestions, like encouraging isolation from others — without recommending human help and often failing to recognize distress.
in other words, evidence shows chatbots risk producing insensitive and harmful responses when users are at their most vulnerable.
while ai can complement human care, it should not replace it. ai could provide meaningful support, detect early signs of crisis, flag suicidal ideation and connect users to professional help — if properly regulated.
but that vision requires accountability. openai and character.ai are setting some age limits on their platforms, for example, but policy-makers have yet to set clear boundaries to prevent harm.