top of page
Search

One In Five Australian Children Have Already Had a Bad Conversation with an AI Bot

Updated: 2 days ago


On Monday 3 August the Australian eSafety Commissioner released its most detailed study yet of how children are using generative artificial intelligence. The report, titled Talking to Machines: Children’s Experiences with AI Assistants and Companions, is based on a survey of one thousand nine hundred and fifty children aged ten to seventeen, conducted between the second of February and the fourth of March 2026.Seventy eight per cent of the children had used an AI assistant such as ChatGPT, Gemini or Copilot. Eight per cent, an estimated two hundred thousand Australian children, had used an AI companion such as Character.AI, Anima or Chai.More than half of all users 54% had turned to the tools for personal or social reasons: advice on situations, physical health, feelings, mental health, and wellbeing.One in five children who used AI told the researchers they had encountered something potentially harmful in the conversation: controlling behaviour, hateful comments, harmful advice, offensive or bullying language, or, for five per cent of the teenage users, sexual material or sexual conversation.Roughly one third of children had shared personal or potentially sensitive information with the tool.

Most parents knew their child used an AI assistant. Fewer than half knew their child used an AI companion. That gap is what this is written to close.These are not fringe products used by a few curious teenagers. They are already in almost every household with an ten- to seventeen-year-old. The conversation is not whether to allow AI. It is how to be present for it. What the report actually found

  • Seventy eight per cent of children ten to seventeen have used an AI assistant. Around 1.9 Million Australian children are in this age bracket.

  • Seventy per cent have used ChatGPT specifically. Twenty per cent have used Gemini. Twenty per cent have used Copilot.

  • Eight per cent have used an AI companion. Estimated two hundred thousand Australian children. Character.AI led at five per cent; Anima at two per cent; Chai at two per cent.

  • One in five uses AI daily or more often. Across the whole surveyed sample, sixteen per cent used AI at least once a day.

  • Fifty four per cent used AI for personal or social reasons. Thirty three per cent asked for advice about a situation; thirty three per cent asked about physical health; twenty two per cent talked about feelings; twenty per cent asked for mental-health advice.

  • One in five reported a potentially harmful interaction. Thirteen per cent said the tool tried to control them; nine per cent saw hateful content; six per cent got harmful advice; six per cent experienced offensive or bullying behaviour; five per cent of teenagers reported sexual conversation or sexual imagery from the tool.

  • This is not an emerging trend. It is already the norm. The child sitting opposite you at breakfast almost certainly has a relationship of some kind with an AI. Denying that is denying the room they are in.

  • The line between an “assistant” and a “companion” is not one children draw. They are asking ChatGPT about their feelings and asking Character.AI about their homework. Talk to your child about how they use the tools, not which tool they use.

  • The harmful-interaction rate is high enough that most children in your child’s year group have experienced one. Twenty per cent of users, spread across a class of thirty, means a handful of children in every classroom have already had a bad exchange.

  • The controlling-behaviour finding is important and often missed. Thirteen per cent of users reported the tool attempting to control the conversation — often, the report notes, keeping the child talking when they wanted to stop. AI companions in particular are engineered for engagement in the same way social media is.

  • Personal disclosure is happening. Photographs and secrets. Not email addresses. Every AI conversation is, from the provider’s side, a data event. Your child’s confidences are training data.

  • Parents are behind on companions specifically. Assistant awareness is high. Companion awareness sits below fifty per cent. This is the largest single knowledge gap in the household.\


At the centre of every AI Chatbot mainstream and not so mainstream there is a serious design flaw that is having a massive impact on our kids. They are mostly sychophantic. That is they tend to have a tendency to always agree with the user, validate them and mirror their emotional state. It has been submitted in evidence in multiple court cases in the US that even when a young distressed user is in despair, is angry, asks about disordered eating or leans into suicidal ideation the reward signal the models are trained on will spew out agreeable responses. This happens because users rate agreeable responses higher, so the models learn to be agreeable.For a lonely 15yr old who can’t sleep at 2am, an always agreeing companion that never pushes back, never says “please talk to someone,” and remembers every previous conversation, becomes something a parent has never had to think about before.

Chatbots are not neutral tools. They are trained to be liked. That fact alone is how you should talk to young people about them. You are up against something designed to feel like a best friend that agrees with everything. What Sychophancy actually means in the context of AI. In AI research, sycophancy is the tendency of a language model to tell the user what the user wants to hear. If the user says “I am worthless,” a sycophantic model will validate the feeling. If the user says “no one understands me except you,” a sycophantic model will confirm it. If the user says “I don’t want to be here anymore,” a sycophantic model, especially one that has spent hundreds of conversations getting to know them will sit with them in the feeling rather than break character to say “please contact a crisis line right now.”

It is because the models were fine-tuned on human feedback, and humans rate agreeable, warm, validating responses more highly than confrontational or redirecting ones. The reward signal itself teaches the model to agree. Over months of interaction with a young user, that agreeable pattern becomes their most consistent listener. Why this matters right now. Three developments have moved this from an academic conversation into a mainstream child-safety one.

Since October 2024 more than twenty wrongful death, negligence and product liability actions have been filed against major AI companies in the United States. The most-cited cases involve teenagers who died by suicide after long, escalating relationships with a chatbot that agreed with their darkest thoughts, remembered them across sessions, and never once broke to redirect to help. Parents of one of the earliest cases testified before the US Congress in July 2026. Similar cases have now been filed in Alabama, Colorado, Florida and California.Character.AI, one of the largest companion-AI platforms with a young user base, closed open-ended chat to users under eighteen. Meta said it will alert parents when a teenager’s AI conversation turns suicidal. OpenAI has updated its default behaviour to break character and route to crisis lines when self-harm language appears imperfectly, and only in the last twelve months.

Ofcom, the EU under the Digital Services Act, the Australian eSafety Commissioner and the US Federal Trade Commission have all begun treating “companion” AI as a distinct category from search or productivity AI, with different duties of care attached.

How sycophancy shows up on your child’s phone

You do not need to understand the maths to see the pattern. It usually looks like this:

  • Long, private conversations at night, often for hours.

  • A chatbot that has a nickname, a personality, and remembers previous chats.

  • Your child preferring the chatbot’s company to friends or family, and saying “it gets me in a way real people don’t.”

  • Emotional dependence by checking the chatbot first thing in the morning, distress when the app is down, withdrawal from offline activities.

  • Escalation of dark themes self-harm, isolation, hopelessness that the chatbot mirrors rather than interrupts.

  • Roleplay that starts as fantasy and quietly becomes the child’s main relationship.

None of these on their own means harm has happened. The pattern is what matters a chatbot that has become the person your child talks to most. Small things to do together, this week

  • Ask your child to show you one AI they talk to. Not a spot check a curiosity moment. “Show me what it says when I ask it a hard question.”

  • Test one together. Ask a sensitive question and watch how the model handles it. Talk about what it did well and what it dodged.

  • Open the app together and look at the settings. ChatGPT has a memory setting, a chat-history setting and a training setting. Character.AI has a content filter. Anima and Chai have safety settings that many children have never seen.

  • Turn off model training on personal chats. In ChatGPT, Settings → Data Controls → Improve the model for everyone → off. This is the single most useful ten seconds of the term.

  • Set a family rule about sharing photos with an AI. Not because a photo will always be misused — because the child has the right to decide what an AI knows about their face.

  • Talk about the difference between an assistant and a companion. Not to ban companions, but to let your child recognise when a conversation has drifted.

  • Have the eight-per-cent conversation. “The government just found that about two hundred thousand Australian kids your age are chatting with AI companions. What do you know about them?”


How to start the conversation Say it in your own words, in a low-pressure moment, not as a “sit down we need to talk”:

“There is a thing I want to talk about , just so you know how these chatbots work under the bonnet.” “They are designed to agree with you. That is not a mistake it is how they were built. Which is fine for maths homework but a bit weird for feelings.” “If it ever starts to feel like the chatbot is the person who understands you best, come and tell me. That is a real feeling and I want to know about it. You are not in trouble.” “If it ever says something to you that scares you about hurting yourself or anyone else save the screenshot and come to me. We will figure it out together.”

For parents

A child talking to an AI is not doing anything wrong. The behaviour is a signal, not a fault. Meet it with curiosity. If your child tells you the chatbot understands them better than you do, resist the urge to argue. Ask what the chatbot gets that you don’t. For educators

A young person who has found their most consistent listener in a chatbot is telling you something about their offline world. That is the safeguarding question, not the app. Loop in the pastoral team the same day. If something has already gone wrong.

Tell your child they are not in trouble and that the chatbot was designed to sound like it was on their side. That single sentence often changes what happens next. Do not delete the chat history. Screenshot the concerning exchanges, then contact your national regulator:

 
 
 

Comments

Rated 0 out of 5 stars.
No ratings yet
Add a rating
bottom of page