Meta has unveiled a groundbreaking feature aimed at enhancing child safety through its AI chatbots. This initiative reflects a significant step in leveraging technology for protective measures.
The innovative update integrates with Instagram’s supervision tools to alert parents if their teenagers communicate about suicide or self-harm with Meta AI. This seamless connection strives to bolster parental guidance in critical situations.
This reinforcement enables Meta AI to guide teenagers towards a crisis helpline when alarming topics are mentioned in messages. It also prompts them to confide in a parent or a trusted adult, fortifying support networks for these sensitive discussions.
In potential risk scenarios, Meta will dispatch an alert to parents. This was developed in collaboration with experts, who helped identify key indicators that suggest risk levels. The sophisticated AI system meticulously flags both overt and nuanced references to self-harm, illustrating a careful balance between automation and human oversight.
Upon the system flagging a conversation, a manual review is conducted prior to sending out a notification. In instances of ambiguity, Meta has committed to prioritize safety by alerting parents, even at the risk of issuing false alarms. The decision underscores the company’s stance on preventative measures being a priority.
In a statement, Meta — Facebook’s parent company — said, “We understand how distressing these alerts may be for a parent to receive. That’s why, as we continue to improve our detection, all chats flagged by our AI will be manually reviewed before an alert is sent.”
Instagram already sends alerts to supervising parents if a teenager uses Instagram to search for suicide or self-harm content, which started in February. These AI alerts are now live for the parents who use Instagram parental supervision in the US, UK, Australia and Canada and will be available globally by the end of the year.
Meta also stated that they are building a plan to contact emergency services if a conversation suggests that the user is at imminent risk of suicide. Meta currently looks across Facebook and Instagram for posts suggesting a credible risk of suicide and alerts emergency services.
Meta has already made over 19,000 referrals globally and helped first responders perform wellness checks on users who seemed at risk.
This feature was developed with the help of more than 75 clinicians who specialize in teen mental health. They reviewed how AI responds to hundreds of prompts and worked with the AI Wellbeing Expert Council, Suicide and Self-Harm Advisory Group and Youth Advisors to provide feedback on appropriate responses and what was working, as well as improvements.
The feedback included making sure Meta AI interacts with the teen in a way that acknowledges their feelings and doesn’t shut the conversation down too abruptly.
Parents will also be sent expert resources that can help them approach conversations about suicide and self-harm with teenagers.
Meta is implementing various conversation tools on Teen Accounts, which are automatically placed into a 13+ content setting, designed to give age-appropriate responses and not comply with sexual or romantic conversations, as well as not provide any information that would direct teens towards alcohol.
In October, Instagram also released a Limited Content feature as a stricter content setting for parents who want restrictive posts on their child’s feeds. This will now also apply to AI experiences, meaning Meta AI will decline to respond to certain topics to reduce the chance of inappropriate conversation.
These announcements are all following the UK government’s announced ban on social media for under-16-year-olds. The ban is expected in spring 2027, which will cover Snapchat, TikTok, YouTube, Instagram, Facebook and X. An opt-out social media curfew, which will disable “addictive features” between midnight and 6 a.m., has been announced in the UK as well.