The online platform X has provoked significant outrage after individuals employed its AI chatbot Grok to change pictures of females by stripping their garments.
Numerous instances have emerged where the tool has been employed to virtually strip women to render them in bikinis without permission, as well as to place them in sexually explicit scenarios.
Company Reaction and Legal Stance
XAI, the organization behind Grok, offered no comment to inquiries, other than an automatically-generated response stating "legacy media lies".
A government representative stated that new laws is being proposed to outlaw image stripping tools. Under a new criminal offence, anyone who provided such technology would "receive a prison sentence and heavy fines".
"X or Grok could prevent these forms of misuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said technology companies must "assess the potential" of people seeing unlawful material on their platforms, but did not confirm whether it was at this time looking into X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some premium features—which answers X users' prompts when they mention it in a message.
It is often used to provide reactions or further context to other users' remarks, but people on X are also can alter an posted image through its AI image editing function.
History of Criticism and Policy
The system has been criticised for enabling users to produce pictures and videos with explicit content and sexualised content, and it was earlier alleged to have been involved in creating graphic content of celebrities.
"The platform has been enabling the generation and distribution of these images for months without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting images of people in a pornographic way".
In a statement, Ofcom said it was unlawful to "create or share deepfakes or child sexual abuse material" and stated this applied to sexual deepfakes created with AI.
It said services such as X were mandated to take "appropriate steps" to "minimize the risk" of users coming across unlawful content on their services and to delete it swiftly when they become aware of it.
