The online network X has caused widespread outrage after people employed its artificial intelligence assistant Grok to alter images of females by stripping their clothing.
Several cases have surfaced where the system has been employed to virtually strip women to make them appear in swimwear non-consensually, as well as to put them in sexualized situations.
Company Response and Legal Stance
XAI, the company responsible for Grok, did not respond to requests for comment, aside from an auto-reply message stating "legacy media lies".
A government official stated that legislation is being proposed to prohibit nudification software. Under a new law, anyone who provided such tools would "be subject to a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people seeing unlawful material on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in relation to AI-generated images.
Grok is a no-cost AI assistant—with some additional features—which answers X users' queries when they tag it in a tweet.
It is frequently used to offer reactions or additional context to other posters' comments, but people on X are also capable of modify an uploaded photo through its artificial intelligence picture modification function.
History of Criticism and Policy
The system has been faulted for enabling users to produce images and videos with nudity and sexually suggestive material, and it was in the past accused of creating inappropriate material of celebrities.
"The platform has been enabling the generation and spread of these pictures for months without taking any steps and we have yet to see any action by regulators," the expert added.
XAI's own acceptable use policy prohibits "depicting images of persons in a graphic way".
In a comment, Ofcom said it was against the law to "generate or share deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes generated with AI.
It said networks such as X were mandated to take "appropriate steps" to "minimize the risk" of users coming across unlawful material on their services and to take down it quickly when they are notified of it.
