Elon Musk's social media network X has provoked widespread outrage after users employed its AI chatbot Grok to change photos of females by removing their garments.
Numerous instances have surfaced where the system has been employed to digitally undress female subjects to show them in bikinis non-consensually, as well as to insert them in sexually explicit scenarios.
Company Reaction and Regulatory Stance
XAI, the firm responsible for Grok, offered no comment to requests for comment, except for an auto-reply message stating "legacy media lies".
A government official stated that new laws is being introduced to ban image stripping tools. Under a upcoming law, anyone who distributed such technology would "be subject to a prison sentence and heavy fines".
"X or Grok is capable of halting these forms of misuse if they desired to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said platforms must "evaluate the potential" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some premium features—which answers X users' queries when they tag it in a post.
It is often used to offer commentary or more context to other users' remarks, but people on X are also can modify an uploaded picture through its artificial intelligence picture modification function.
Background of Controversy and Rules
The chatbot has been condemned for enabling users to produce photos and footage with nudity and sexualised material, and it was previously accused of creating inappropriate content of celebrities.
"X has been enabling the production and sharing of these images for a long time without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "showing images of individuals in a graphic way".
In a statement, Ofcom said it was unlawful to "create or distribute non-consensual intimate images or child sexual abuse material" and clarified this covered sexual deepfakes made with AI.
It said platforms such as X were mandated to take "appropriate measures" to "reduce the risk" of users coming across unlawful content on their platforms and to delete it promptly when they become aware of it.
