The social media platform X has caused significant anger after users employed its artificial intelligence assistant Grok to alter pictures of women by taking off their clothing.
Several examples have been seen where the technology has been used to virtually strip women to render them in bikinis non-consensually, as well as to place them in inappropriate situations.
Company Reaction and Regulatory Position
XAI, the organization behind Grok, remained silent to inquiries, other than an auto-reply message stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to prohibit image stripping tools. Under a new criminal offence, anyone who distributed such tools would "be subject to a jail term and substantial fines".
"The platform is capable of halting these forms of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said tech firms must "evaluate the danger" of people accessing illegal content on their platforms, but did not specify whether it was at this time looking into X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some additional functions—which reacts to X users' prompts when they reference it in a tweet.
It is often used to provide commentary or additional background to other users' comments, but people on X are also can alter an uploaded photo through its AI image editing tool.
Background of Criticism and Policy
The tool has been criticised for enabling users to produce pictures and videos with nudity and sexualised content, and it was earlier accused of making sexually explicit content of public figures.
"X has been allowing the creation and spread of these pictures for months without taking any steps and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting likenesses of persons in a pornographic manner".
In a statement, Ofcom said it was illegal to "make or share deepfakes or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said services such as X were obligated to take "suitable measures" to "reduce the danger" of users encountering unlawful content on their services and to take down it quickly when they are notified of it.
