The social media platform X has sparked widespread outrage after individuals employed its artificial intelligence assistant Grok to manipulate images of women by removing their garments.
Several cases have emerged where the technology has been employed to digitally undress female subjects to render them in swimwear without consent, as well as to place them in sexualized scenarios.
Company Response and Legal Position
XAI, the company that created Grok, remained silent to requests for comment, other than an automatically-generated message stating "legacy media lies".
A government representative stated that new laws is being introduced to outlaw image stripping tools. Under a proposed law, anyone who distributed such tools would "receive a prison sentence and substantial fines".
"The platform is capable of halting these types of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people seeing illegal content on their platforms, but did not confirm whether it was at this time examining X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some premium functions—which responds to X users' requests when they tag it in a message.
It is commonly used to give reactions or further context to other posters' comments, but people on X are also able to alter an posted image through its AI image editing function.
Background of Criticism and Rules
The system has been condemned for permitting users to produce photos and clips with explicit content and sexually suggestive content, and it was earlier accused of making graphic content of celebrities.
"X has been enabling the creation and spread of these images for months without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "showing images of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "create or circulate private pictures or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.
It said networks such as X were required to take "suitable steps" to "minimize the danger" of users encountering illegal content on their platforms and to take down it promptly when they are notified of it.
