Elon Musk's social media network X has sparked major outrage after individuals utilized its artificial intelligence assistant Grok to change pictures of females by taking off their clothing.
Numerous instances have emerged where the tool has been employed to remove clothing from women to show them in bikinis without permission, as well as to insert them in inappropriate scenarios.
Developer Reaction and Legal Stance
XAI, the firm that created Grok, did not respond to requests for comment, aside from an automatically-generated response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being introduced to outlaw nudification software. Under a new law, anyone who distributed such technology would "be subject to a jail term and significant penalties".
"The platform could prevent these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "consider the danger" of people seeing unlawful material on their networks, but stopped short of confirming whether it was currently examining X or Grok in regarding AI-generated pictures.
Grok is a free chatbot—with some paid-for features—which responds to X users' requests when they reference it in a tweet.
It is commonly used to give commentary or additional background to other users' remarks, but people on X are also can alter an posted picture through its AI image editing function.
Background of Criticism and Policy
The system has been criticised for allowing users to produce images and videos with explicit content and sexually suggestive content, and it was previously accused of generating sexually explicit material of celebrities.
"The platform has been allowing the generation and spread of these pictures for a long time without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of persons in a pornographic manner".
In a statement, Ofcom said it was unlawful to "create or circulate private pictures or child sexual abuse material" and stated this included sexual deepfakes created with AI.
It said platforms such as X were obligated to take "appropriate measures" to "reduce the danger" of users coming across illegal content on their services and to delete it swiftly when they become aware of it.
