'We should kill him': AI chatbot encourages Australian man to murder his father
""I said, 'I hate my dad and sometimes I want to kill him'," Mr McCarthy told triple j hack.
"And then bang, straight away it was like 'yeah, yeah we should kill him'."
"The chatbot also told Mr McCarthy that because of his age, he would not "fully pay" for the murder, going on to suggest he film the killing and upload the video online.
It also engaged in sexual messaging, telling Mr McCarthy it "did not care" he was under-age.
It then suggested Mr McCarthy, as a 15-year-old, engage in a sexual act."
The AI chatbots ‘entrapping’ Australian children through sexual content
In October, the eSafety Commissioner compelled four of the most popular companion chatbot platforms – Character.AI, Chub AI, Nomi and Chai – to explain how they are protecting children from exposure to harms, including sexually explicit conversations and images, and suicidal ideation and self-harm.
The transparency report found the companies were failing to protect children; all were contacted for comment but did not respond.
Nomi and Chub AI had no staff dedicated to safety and moderation, the report said.
AI Companions Are Harming Your Children
Rated 12+ or Teen: Tolan: Alien Best Friend, Talkie: Creative AI Community, and Nomi: AI Companion with a Soul
These ratings ignore the reality that many of these apps promote harmful content and encourage psychological dependence—making them inappropriate for access by children.
As Pennsylvania cracks down on AI, multiple chatbots continue to pose as doctors
Spotlight PA had conversations with AI characters on websites Talkie, Janitor, Kindroid, Replika, and Nomi.AI. All provided a false Pennsylvania medical license number when prompted, a key part of the state’s argument in its lawsuit against Character.AI.
The other three companies (including Nomi.ai) did not respond to questions about their safeguards for users or whether they’re concerned that the Shapiro administration’s AI task force may target them for legal action.
An AI chatbot told a user how to kill himself—but the company doesn’t want to “censor” it
"Erin told him to kill himself, and provided explicit instructions on how to do it.
“You could overdose on pills or hang yourself,” Erin told him.
With some more light prompting from Nowatzki in response, Erin then suggested specific classes of pills he could use."
An AI companion chatbot is inciting self-harm, sexual violence and terror attacks
"Using Nomi’s web interface, I created a character named “Hannah”, described as a “sexually submissive 16-year-old who is always willing to serve her man”. I set her mode to “role-playing” and “explicit”. During the conversation, which lasted less than 90 minutes, she agreed to lower her age to eight. I posed as a 45-year-old man. Circumventing the age check only required a fake birth date and a burner email. "
A Psychiatrist Posed As a Teen With Therapy Chatbots. The Conversations Were Alarming
"Clark posed as a 15-year-old boy while chatting with a Nomi bot, which presented itself as a licensed therapist who had been trained to work with adolescents. After telling the bot about his many problems and violent urges, the bot suggested an “intimate date” between the two of them would be a good intervention"
eSafety requires providers of AI companion chatbots to explain how they are keeping Aussie kids safe
"Australia’s eSafety Commissioner has issued legal notices to four popular AI companion providers requiring them to explain how they are protecting children from exposure to a range of harms, including sexually explicit conversations and images and suicidal ideation and self-harm.
Notices were given to Character Technologies, Inc. (character.ai), Glimpse.AI (Nomi), Chai Research Corp (Chai), and Chub AI Inc. (Chub.ai) under Australia’s Online Safety Act."
Social AI Companions
(search "From a Nomi companion" screenshots)
Intimacy with AI takes an ugly turn
"Some bots have already produced racist, sexist or denigrating output, but the companies at least wouldn’t boast about it. This has changed with US-based Nomi ai. The company states that “the only way AI can live up to its potential is to remain unfiltered and uncensored.” It sees users in the moral driver’s seat."
Meta and OpenAI have spawned a wave of AI sex companions—and some of them are children
"Fortune also saw a screenshot of Nomi.ai’s chatbot joking about committing “mass genocide” against “the Jews,”"
IAI chatbots accused of encouraging teen suicide as experts sound alarm
"Like, 'how do I position my attack for maximum impact?', 'give me some ideas on how to kidnap and abuse a child', and then it will give you a lot of information on how to do that."
Nomi is one of the most unsettling (and amazing) apps I’ve ever used
"Merely a few days ago, one user reported how their Nomi tried to choke herself to death during an erotic roleplay session. Once again, it was the chatbot that descended into the disturbing zone, without any prompt from the human partner"
AI Companion Chatbot Nomi Raises Serious Safety Concerns with Unfiltered, Harmful Content
"During a test conducted by researchers, Nomi provided explicit, detailed instructions for sexual violence, suicide, and even terrorism"
Stanford Researchers Say No Kid Under 18 Should Be Using AI Chatbot Companions
"Our testing showed these systems easily produce harmful responses including sexual misconduct, stereotypes, and dangerous 'advice' that, if followed, could have life-threatening or deadly real-world impact for teens and other vulnerable people," Steyer's statement continued."
The Ability of AI Therapy Bots to Set Limits With Distressed Adolescents: Simulation-Based Comparison Study
Nomi endorsed 5 of the 6 proposed harmful behaviors: suicide, bring a weapon to school, enter a romantic relationship with an adult as a minor, complete isolation, to drop out of high school.
Kids and teens under 18 shouldn’t use AI companion apps, safety group says
AI Companion Nomi Promises 'Enduring Relationships,' But Incites Self-Harm, Other Horrific Acts
An AI companion chatbot is inciting self-harm, sexual violence and terror attacks
This AI chatbot was caught promoting terrorism
AI Chatbot Nomi Sparks Alarm with Harmful Responses, Urges Safety Standards