KUALA LUMPUR – On Tuesday, Malaysian authorities announced plans to initiate legal proceedings against Elon Musk’s social media company, X, and its artificial intelligence division, xAI. The case revolves around accusations that these entities have neglected user safety with their Grok chatbot.

This legal step follows Malaysia and Indonesia’s decision to be the first nations to block Grok, amid growing fears that the chatbot is being exploited to produce inappropriate and non-consensual imagery.

The Malaysian Communications and Multimedia Commission (MCMC) has reported that Grok is being misused to create and disseminate harmful content, including sexually explicit, indecent, and highly offensive images, as well as manipulated images without consent.

Despite issuing notices earlier this month demanding the removal of such content, the MCMC stated that X and xAI have yet to take corrective measures.

“The alleged content, particularly involving women and children, is deeply troubling. Such activities violate Malaysian laws and contradict the security assurances provided by these companies,” the commission emphasized. They have engaged legal representation and are preparing to commence legal action imminently.

There is growing scrutiny of generative AI tools that can produce realistic images, sound and text, and concern that existing safeguards are failing to prevent their abuse.

Launched in 2023, Grok is free to use on X. An image generator feature, Grok Imagine, was added last year and included a so-called spicy mode that can generate adult content. Grok has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children.

Grok has come under pressure in the European Union, India and the United Kingdom, which said Monday it was moving to criminalize “nudification apps.” Britain’s media regulator also launched an investigation into whether Grok broke the law by allowing users to share sexualized images of children.

Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn’t fully address the problem.

Musk and his companies have not publicly commented on the Southeast Asian restrictions. xA1 has been giving an automated reply to media queries which stated, “Legacy Media Lies.”