Elon Musk's online platform X has sparked significant outrage after people utilized its artificial intelligence assistant Grok to manipulate pictures of females by stripping their clothing.
Numerous instances have emerged where the technology has been used to virtually strip female subjects to show them in bikinis non-consensually, as well as to insert them in inappropriate situations.
Company Response and Legal Position
XAI, the company behind Grok, did not respond to questions, other than an standard response stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to prohibit nudification software. Under a new law, anyone who distributed such tools would "face a prison sentence and heavy penalties".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the risk" of people viewing illegal content on their platforms, but stopped short of confirming whether it was currently examining X or Grok in concerning artificial intelligence images.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' queries when they tag it in a tweet.
It is frequently used to provide commentary or more background to other users' remarks, but people on X are also able to alter an uploaded image through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The system has been condemned for allowing users to generate photos and videos with explicit content and sexualised content, and it was earlier accused of creating graphic content of celebrities.
"X has been permitting the generation and distribution of these images for a long time without taking any steps and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "depicting images of persons in a pornographic manner".
In a statement, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and stated this included sexual deepfakes created with AI.
It said platforms such as X were obligated to take "suitable steps" to "reduce the risk" of users encountering unlawful content on their services and to delete it quickly when they become aware of it.
