The online platform X has sparked significant outrage after individuals employed its AI chatbot Grok to manipulate pictures of females by stripping their clothing.
Multiple instances have surfaced where the system has been employed to digitally undress female subjects to render them in bikinis non-consensually, as well as to put them in inappropriate scenarios.
Developer Reaction and Legal Stance
XAI, the organization behind Grok, remained silent to questions, except for an automatically-generated response stating "legacy media lies".
A Home Office official stated that new laws is being introduced to outlaw nudification technology. Under a new criminal offence, anyone who supplied such tools would "be subject to a jail term and heavy fines".
"X or Grok is capable of halting these types of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Capabilities
The regulator Ofcom said tech firms must "assess the potential" of people viewing unlawful material on their networks, but did not specify whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some premium features—which responds to X users' prompts when they tag it in a message.
It is often used to give commentary or more context to other posters' remarks, but people on X are also can modify an uploaded picture through its AI image editing function.
Background of Controversy and Policy
The tool has been criticised for allowing users to create images and videos with explicit content and sexualised content, and it was previously alleged to have been involved in making graphic material of celebrities.
"The platform has been allowing the creation and distribution of these pictures for a long time without taking any measures and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "showing likenesses of people in a graphic fashion".
In a comment, Ofcom said it was illegal to "create or distribute private pictures or child sexual abuse material" and clarified this covered explicit AI images made with AI.
It said services such as X were mandated to take "suitable steps" to "lessen the danger" of users coming across illegal content on their platforms and to remove it swiftly when they become aware of it.