The social media network X has caused significant anger after individuals employed its artificial intelligence assistant Grok to change photos of women by stripping their attire.
Multiple cases have surfaced where the tool has been employed to digitally undress women to render them in swimwear without consent, as well as to place them in inappropriate scenarios.
Developer Reaction and Regulatory Stance
XAI, the company behind Grok, remained silent to questions, aside from an standard response stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to ban image stripping software. Under a new law, anyone who provided such technology would "be subject to a prison sentence and heavy penalties".
"The platform is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said platforms must "consider the risk" of people seeing unlawful material on their networks, but did not confirm whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost AI assistant—with some premium functions—which reacts to X users' requests when they reference it in a tweet.
It is often used to give reactions or further context to other users' remarks, but people on X are also able to edit an uploaded photo through its AI image editing function.
Background of Controversy and Rules
The system has been condemned for permitting users to create images and footage with explicit content and sexualised material, and it was previously alleged to have been involved in making sexually explicit content of public figures.
"The platform has been permitting the production and spread of these images for months without taking any action and we have not seen any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy bans "depicting images of people in a graphic way".
In a statement, Ofcom said it was illegal to "generate or distribute deepfakes or child sexual abuse material" and confirmed this covered explicit AI images generated with AI.
It said platforms such as X were required to take "suitable actions" to "lessen the danger" of users encountering illegal material on their platforms and to remove it promptly when they learn of it.
