The increasing prevalence of AI-generated 'people' on platforms such as Instagram has led to the rise of “synthetic relationships”.
A synthetic relationship refers to an interaction or connection between a human user and an artificially intelligent entity, often portrayed as a human-like character or in the case of Facebook, celebrities with a side hustle of being a chatbot. These AI-generated personas, created using advanced technologies, can mimic human behaviors and communication styles, making them seem lifelike. The relationship is termed 'synthetic' because, unlike traditional human-to-human relationships, one side of the interaction is completely artificial and programmed.
These synthetic entities, like the ones captured in the video clips, can appear on social media platforms, chat applications, or in virtual environments and are designed to engage in convincing conversations, respond to emotional cues, and sometimes even form emotional bonds with users.
While these relationships can offer companionship or entertainment, they also raise concerns about the emotional impact on individuals who may find distinguishing between genuine human interaction and AI-generated responses designed to look and act like real humans. This is one of the things we discuss in our eReady Kids course because they pose a unique challenge, especially for young users that need to learn from a young age how to tell the difference. They can create body image issues and set unrealistic expectations of a child comparing their life or looks to something that does not exist.
There's a significant concern that these interactions, which are becoming more sexualised, might be a cover for criminal activities like sextortion by criminal gangs. This involves tricking young users into sharing personal or sensitive information, which can then be exploited for blackmail. The primary issue here is the difficulty in differentiating real human users from AI-generated profiles, which could lead to dangerous situations if not approached with caution and awareness.
During the school holidays, it's a common mistake to think that online risks, especially those related to schools, take a break. Many years of working in this sector have shown me that issues, particularly in group chats, actually increase during this time. Kids have more time to explore online....and they do.
With the emergence of synthetic relationships, deepfake technology abuse and bullying, the online landscape is changing at pace, and parents are struggling to keep up with challenges. Deepfake tech can create realistic but fake images and videos, which can be especially harmful to students. We have had numerous reports of students deepfaking etc other without realising what they are doing is highly illegal and reportable under image-based abuse and bullying laws. For instance, a simple photo or video shared by a student (or their parent on a public Instagram account, for example) could be captured and manipulated into misleading content, leading to bullying and damage to their reputation, not to mention long-term mental trauma.
The trouble with deepfakes is their ability to blur the lines between what's real and what's not, making it challenging for kids to discern the truth and handle the fallout. To combat this, adjusting privacy settings on social media is essential. Parents should ensure their holiday photos and posts, particularly those involving their children, are shared within a close-knit circle only. This reduces the risk of these images getting screenshots into the wrong hands and being used to create deepfakes to bully them by another student, or by someone else who may create CSAM (Child Sexual Abuse Material) from the image.
Now is also an opportune time for parents to talk to their children about online safety. These conversations should be non-judgemental and non-confrontational and highlight the need to be careful about what they share, the dangers to their digital footprint, and the realities of technologies like deepfakes and synthetic relationships. It's crucial for children/teenagers to understand that the risks of the online world persist, even outside of school, and they need to be equipped to navigate these challenges.
Parents need to be prepared for when things go wrong. This includes recognizing signs of cyberbullying, understanding how to report and remove harmful content, and maintaining open lines of communication with their children to discuss any online issues that arise. In this constantly changing environment, parental awareness and proactive engagement are key to safeguarding children during the holidays and beyond