KUALA LUMPUR, Malaysia (AP) — Malaysian authorities said Tuesday they will take legal action against Elon Musk’s social media platform X and its artificial intelligence unit xAI, accusing the companies of failing to ensure the safety of users of its Grok chatbot.
The move came just days after Malaysia and Indonesia became the first countries to block access to Grok, as concerns grow that it is being misused to generate sexually explicit and nonconsensual images.
The Malaysian Communications and Multimedia Commission said in a statement that it has identified the misuse of Grok to generate and distribute harmful content including sexually explicit, indecent, extremely offensive as well as non-consensual manipulated images.
It said it served notices to X and xAI this month to remove the harmful content but no action has been taken.
“Content allegedly involving women and children is a matter of great concern. Such conduct is against Malaysian law and undermines the security commitments” stated by the companies, it said. The commission has appointed a lawyer and said legal proceedings would begin soon.
There is growing scrutiny of generative AI tools that can produce realistic images, sound and text, and concern that existing safeguards are failing to prevent their abuse.
Launched in 2023, Grok is free to use on X. An image generator feature, Grok Imagine, was added last year and included a so-called spicy mode that can generate adult content. Grok has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children.
Grok has come under pressure in the European Union, India and the United Kingdom, which said Monday it was moving to criminalize “nudification apps.” Britain’s media regulator also launched an investigation into whether Grok broke the law by allowing users to share sexualized images of children.
Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn’t fully address the problem.
Musk and his companies have not publicly commented on the Southeast Asian restrictions. xA1 has been giving an automated reply to media queries which stated, “Legacy Media Lies.”