Elon Musk's social media network X has provoked significant anger after individuals utilized its AI chatbot Grok to manipulate pictures of women by taking off their garments.
Numerous cases have been seen where the tool has been used to digitally undress female subjects to make them appear in swimwear non-consensually, as well as to insert them in sexually explicit situations.
Company Reaction and Legal Position
XAI, the company responsible for Grok, remained silent to inquiries, other than an standard message stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to ban nudification technology. Under a new criminal offence, anyone who distributed such technology would "face a jail term and significant penalties".
"The platform has the ability to stop these forms of abuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "consider the potential" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some additional features—which responds to X users' queries when they mention it in a message.
It is often used to provide commentary or additional background to other users' remarks, but people on X are also can alter an uploaded picture through its AI image editing tool.
History of Criticism and Policy
The system has been criticised for allowing users to generate photos and videos with explicit content and sexualised content, and it was in the past alleged to have been involved in making graphic material of public figures.
"X has been permitting the production and distribution of these images for a long time without taking any steps and we have not seen any challenge by regulators," the expert added.
XAI's own acceptable use policy bans "showing likenesses of persons in a graphic manner".
In a comment, Ofcom said it was against the law to "make or circulate deepfakes or illegal content" and clarified this covered sexual deepfakes generated with AI.
It said networks such as X were mandated to take "necessary actions" to "reduce the risk" of users encountering unlawful material on their platforms and to remove it promptly when they are notified of it.
