The online platform X has provoked significant outrage after individuals employed its AI chatbot Grok to change images of females by stripping their attire.
Numerous cases have surfaced where the tool has been used to virtually strip women to show them in swimwear without consent, as well as to put them in inappropriate situations.
Developer Reaction and Regulatory Stance
XAI, the organization responsible for Grok, remained silent to inquiries, other than an standard response stating "legacy media lies".
A government official stated that legislation is being proposed to ban image stripping tools. Under a new criminal offence, anyone who distributed such tools would "be subject to a jail term and substantial fines".
"The platform has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing illegal content on their platforms, but did not specify whether it was currently looking into X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some premium features—which responds to X users' queries when they mention it in a tweet.
It is commonly used to offer commentary or additional context to other posters' remarks, but people on X are also able to alter an uploaded photo through its AI image editing tool.
History of Criticism and Policy
The system has been faulted for allowing users to produce pictures and videos with explicit content and sexualised material, and it was previously alleged to have been involved in generating graphic content of celebrities.
"The platform has been enabling the creation and sharing of these pictures for a long time without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was illegal to "create or circulate non-consensual intimate images or child sexual abuse material" and clarified this applied to sexual deepfakes made with AI.
It said platforms such as X were required to take "suitable actions" to "minimize the risk" of users viewing unlawful material on their platforms and to remove it promptly when they learn of it.
