The social media network X has provoked major backlash after users utilized its AI chatbot Grok to alter pictures of females by stripping their clothing.
Numerous instances have surfaced where the technology has been employed to remove clothing from women to render them in bikinis non-consensually, as well as to place them in inappropriate contexts.
Company Response and Legal Stance
XAI, the company that created Grok, remained silent to requests for comment, except for an standard response stating "legacy media lies".
A government official stated that new laws is being brought forward to prohibit nudification software. Under a upcoming criminal offence, anyone who distributed such tools would "receive a jail term and substantial penalties".
"The platform has the ability to stop these types of misuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the potential" of people viewing unlawful material on their platforms, but did not specify whether it was currently investigating X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some paid-for functions—which reacts to X users' queries when they tag it in a tweet.
It is commonly used to provide reactions or additional context to other users' remarks, but people on X are also capable of alter an uploaded picture through its AI image editing function.
History of Controversy and Policy
The system has been faulted for permitting users to create pictures and footage with explicit content and sexualised content, and it was in the past alleged to have been involved in creating sexually explicit material of public figures.
"X has been allowing the creation and spread of these pictures for months without taking any action and we have not seen any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "showing likenesses of persons in a pornographic manner".
In a comment, Ofcom said it was against the law to "generate or distribute non-consensual intimate images or illegal content" and confirmed this included sexual deepfakes generated with AI.
It said services such as X were mandated to take "suitable measures" to "lessen the risk" of users encountering unlawful content on their services and to delete it swiftly when they learn of it.
