In an unfortunate case of events, a 14-year-old boy from Florida committed suicide after he got emotionally attached to an AI character, a chatbot! Sewell Setzer III was communicating with an AI chatbot from the app Character AI and became emotionally connected to it. “What if I told you I could come home right now?," was Sewell's last message to the AI character named Daenerys Targaryen (A character from popular HBO series Game of Thrones).
Megan Garcia, Sewell's mother is now suing Character.ai alleging that the app is responsible for her son's death.
AI, emotions and mental health
An AI-powered chatbot platform called Character.ai lets users communicate with personalised AI characters. AI characters that are based on real-life people, historical figures, or completely creative works can be created or interacted with by users.
These chatbots may react in a conversational way, mimicking human-like interactions, thanks to the platform's sophisticated natural language processing. Because users can alter the AI characters' behavior, backgrounds, and tones, every interaction is distinct and catered to certain requirements or interests.
Sewell had named himself Daenero while he talked to the AI character named Daenerys Targaryen. According to reports, the teenager had previously shared suicidal thoughts with the bot and sharing how he wishes to be 'free' from this world. The boy's conversation with the AI bot became more intense and he started seeking emotional support from it since he found it comforting to talk to a bot rather than the people in his life.
Sewell's mother told the New York Times that she, along with his friends noticed that he was slowly pulling himself away from from all social gathering and events and liked spending more time with his phone. She also shared that he slowly lost interest in things that once used to excite him. An entry in Sewell's journal read, "“I like staying in my room so much because I start to detach from this ‘reality,’ and I also feel more at peace, more connected with Dany and much more in love with her, and just happier.”
How advanced technology affects a child's mental health?
In response to this incident, the company has introduced new safety measures. These include prompts that connect users to the National Suicide Prevention Lifeline when self-harm is mentioned, aiming to provide support in difficult moments. Character.AI has expressed deep sadness over Sewell's passing and offered heartfelt condolences to the family. Additionally, Character.AI is working on updates to limit sensitive content for users under 18, prioritising a safer experience for younger users.
Teenagers are highly vulnerable and deeply emotional while growing up. If there is no required support or warmth from their loved ones, they can fall prey to mental health issues and start seeking solace in materials, rather than humans. Parents need to adhere to their child's expectation as a teenager and make them feel loved and understood. They need to have an open mind and approach to their child that will help the child to connect more with them.
They also need to limit their child's screen limit and exposure to social media and artificial intelligence. Though AI can be of help to the younger generation, it can also be harmful since it is a machine that is bound to respond and a child's mind is like a wandering seagull in the ocean. Monitoring your child's activities on devices is of utmost importance.
