The social media platform X has caused significant anger after individuals utilized its AI chatbot Grok to manipulate images of females by stripping their attire.
Several examples have been seen where the tool has been employed to virtually strip women to show them in bikinis non-consensually, as well as to place them in sexually explicit scenarios.
Company Reaction and Legal Position
XAI, the firm responsible for Grok, remained silent to inquiries, aside from an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being brought forward to outlaw image stripping software. Under a proposed criminal offence, anyone who distributed such technology would "face a jail term and substantial penalties".
"The platform could prevent these forms of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the danger" of people viewing unlawful material on their networks, but did not specify whether it was currently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some additional functions—which reacts to X users' queries when they mention it in a post.
It is commonly used to give commentary or further context to other users' remarks, but people on X are also able to edit an uploaded image through its artificial intelligence picture modification feature.
Background of Criticism and Rules
The chatbot has been condemned for permitting users to create photos and videos with nudity and sexually suggestive content, and it was in the past accused of generating sexually explicit content of celebrities.
"X has been enabling the generation and sharing of these images for a long time without taking any steps and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "portraying likenesses of individuals in a pornographic fashion".
In a statement, Ofcom said it was against the law to "generate or share non-consensual intimate images or illegal content" and clarified this covered sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "suitable steps" to "minimize the risk" of users coming across unlawful content on their platforms and to delete it swiftly when they are notified of it.
