🔗 Share this article

The online platform X has provoked major backlash after individuals employed its AI chatbot Grok to change photos of women by taking off their clothing.

Numerous examples have been seen where the technology has been employed to digitally undress women to render them in bikinis non-consensually, as well as to insert them in inappropriate situations.

Company Response and Regulatory Stance

XAI, the firm that created Grok, did not respond to questions, except for an auto-reply response stating "legacy media lies".

A Home Office spokesperson stated that legislation is being brought forward to outlaw nudification software. Under a new criminal offence, anyone who supplied such tools would "be subject to a prison sentence and heavy penalties".

"The platform has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."

Regulator Stance and Platform Capabilities

The communications watchdog Ofcom said tech firms must "consider the danger" of people viewing unlawful material on their networks, but did not specify whether it was presently investigating X or Grok in regarding artificial intelligence pictures.

Grok is a no-cost AI assistant—with some premium features—which responds to X users' prompts when they reference it in a post.

It is commonly used to give reactions or additional background to other posters' remarks, but people on X are also can modify an uploaded image through its artificial intelligence picture modification tool.

History of Criticism and Policy

The tool has been condemned for allowing users to produce photos and clips with nudity and sexualised material, and it was previously alleged to have been involved in generating sexually explicit material of celebrities.

"X has been permitting the creation and sharing of these pictures for months without taking any action and we have yet to see any intervention by oversight bodies," the expert added.

XAI's own usage rules forbids "portraying likenesses of people in a graphic manner".

In a statement, Ofcom said it was unlawful to "create or share private pictures or illegal content" and stated this covered sexual deepfakes made with AI.

It said platforms such as X were obligated to take "necessary measures" to "lessen the risk" of users viewing illegal material on their platforms and to remove it promptly when they learn of it.