When Elon Musk recently announced the release of several new “companions” for users of the AI chatbot Grok on X to interact with, I could imagine some of the discussions I might soon be having at school.
One of the companions, named “Ani”, takes the form of a scantily dressed anime-style girl, and some have already raised concerns about the safeguarding risks she might present.
Yet Ani is part of a bigger trend: the rise of AI “girlfriends”. These are apps that are marketed as emotionally responsive, always-available digital partners. The AI systems they use are designed to simulate a romantic or affectionate relationship.
On the surface, this might seem harmless. However, these apps present a number of hidden dangers, particularly for teenage boys, who might turn to virtual relationships as a substitute for real-life connections.
So what do teachers need to know about these apps?
What are AI ‘girlfriend’ apps?
AI girlfriend apps are by no means niche. Some have hundreds of thousands of downloads. They allow users to create their “perfect” woman, designing and curating everything from skin tone to eye colour and hair colour. Some are ultra realistic, while others offer the option of “waifu” characters, drawn from Japanese anime and manga, in the style of Grok’s Ani.
Beyond catering for specific preferences for hair colour or eye colour, these apps can also cater for much darker preferences. Many promote the fact that they are “NSFW” (not safe for work). For example, Ani has an unlockable feature that allows users to undress her and see her in her lingerie.
In some apps, users can request explicit pictures. Users can be verbally abusive. Harmful and manipulative behaviours are likely to be passively reinforced; the AI is unlikely to push back or establish boundaries.
For young boys who are still forming an understanding of healthy relationships, this can create dangerous misconceptions.
Boys who have a lack of confidence or low self-esteem might find it incredibly daunting to enter the world of dating, but AI girlfriends are programmed to be supportive, flattering and attentive. They remove the complexities of human relationships, along with the fear.
The problem is that navigating those complex human relationships is an important part of development. If young people can opt out of this and retreat to the comfort of an app, I worry that they will grow up with warped expectations of what intimacy and authentic communication look like.
And there is another risk, too. While these apps might be free to download, they all have a company model behind them. Most apps make money through subscriptions, microtransactions or in-app purchases. Users might be asked to upgrade to a paid model after a certain number of interactions or encouraged to purchase add-ons, such as new dialogue options, designs or features.
Vulnerable teenagers, especially those who are neurodivergent, may not recognise the manipulation being used to extract money from them.
The risk of sextortion
All of this feeds into a related issue that teachers should also be aware of: the rise of sextortion. This is the process of coercing individuals into sharing explicit or intimate images or videos of themselves, which are then used to blackmail them.
It’s one of the fastest growing online crimes, with teenage boys often the targets, and it can have a devastating impact. In 2023 Murray Dowey, a Scottish 16-year-old, took his own life after becoming a victim of sextortion.
Advancements in AI are making the crime even harder for victims to spot. Deep fakes can be used to present a realistic person that boys think they are interacting with.
I worry that normalising the use of AI girlfriend apps could further prime boys for the online interactions that lead to sextortion.
What role can schools play in combating these issues?
How can schools help?
Education and awareness are incredibly important. Discussions around online safety and media literacy should be interwoven within the curriculum.
Teachers should reinforce the importance of privacy settings and the dangers of sharing personal information online. They should ensure that students are aware of the financial risks involved and, when it comes to sextortion, the tactics that might be used by perpetrators. The Internet Watch Foundation and Childline have resources that can be used to help here.
The issue of AI girlfriends should be addressed directly. Explain to students that these are not a replacement for genuine relationships. Schools, as well as families, must nurture boys’ self-confidence and self-esteem, which will prevent them from feeling the need to turn to these apps.
Beyond that, schools should ensure that students have a safe space where they can discuss their experiences without fear of judgement. Make sure they know that they will be supported if they report any concerning incidents.
Teachers should be vigilant for any indicators that a student may be a victim of sextortion. This could be a sudden change in behaviour, withdrawal from social interactions or a reluctance to use electronic devices or be online.
Finally, encourage families to engage with the children in their care and to make sure they are involved in the child’s online life.
Spread awareness of potential online dangers through newsletters or workshops. Empower parents with tools like the Report Remove service (which allows individuals to report and request the removal of explicit images that have been shared without consent), and circulate information about the parental control and safety features of common apps, like X.
If families have a better understanding of the online world, risks to look out for and how to stay safe, they have a better chance of monitoring what is going on.
Thomas Michael is a safeguarding lead
You can now get the UK’s most-trusted source of education news in a mobile app. Get Tes magazine on iOS and on Android
