Elon Musk's online network X has provoked significant anger after users used its artificial intelligence assistant Grok to change photos of women by removing their clothing.
Multiple examples have been seen where the technology has been employed to virtually strip female subjects to render them in bikinis non-consensually, as well as to put them in sexualized situations.
Company Reaction and Regulatory Stance
XAI, the organization behind Grok, did not respond to inquiries, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office official stated that legislation is being proposed to outlaw nudification software. Under a new criminal offence, anyone who supplied such tools would "face a prison sentence and heavy fines".
"X or Grok could prevent these forms of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "consider the potential" of people viewing illegal content on their platforms, but did not specify whether it was presently investigating X or Grok in regarding AI-generated pictures.
Grok is a free chatbot—with some paid-for features—which answers X users' queries when they tag it in a post.
It is commonly used to give reactions or additional context to other users' comments, but people on X are also capable of modify an posted image through its AI image editing function.
Background of Criticism and Policy
The tool has been condemned for enabling users to produce images and clips with explicit content and sexualised material, and it was earlier alleged to have been involved in generating inappropriate material of celebrities.
"X has been allowing the production and distribution of these pictures for a long time without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting images of people in a pornographic fashion".
In a comment, Ofcom said it was illegal to "generate or share deepfakes or illegal content" and stated this covered sexual deepfakes created with AI.
It said networks such as X were mandated to take "appropriate actions" to "reduce the danger" of users coming across unlawful content on their platforms and to take down it promptly when they learn of it.
