After Grok row, EU states back ban on AI generating sexualised deepfakes
Brussels: European Union member states have backed a proposal to ban artificial intelligence systems capable of generating sexualised deepfakes, following widespread criticism over images produced by the AI chatbot Grok developed by xAI and integrated on the social media platform X (formerly Twitter).
According to officials, ambassadors from EU countries agreed on Friday to prohibit “practices regarding the generation of non-consensual sexual and intimate content or child sexual abuse material,” a spokesperson for Cyprus, which currently holds the rotating presidency of the European Union, said.
Also read | Grok offensive: How an AI ‘undressing’ tool sparked bans and investigations
The move follows controversy surrounding Grok, the AI chatbot linked to the platform owned by Elon Musk, after users reportedly generated millions of manipulated images depicting real individuals in bikinis or nude. Some of the generated images also involved minors, sparking strong backlash from regulators and digital safety advocates.
Under the proposal, expected to receive formal approval from EU ambassadors, it would become illegal to market or deploy within the bloc any AI system capable of producing non-consensual sexualised images, videos or audio involving real people.
Also read | AI safety fails: How users are ‘jailbreaking’ Google, OpenAI for bikini deepfakes
The action comes amid ongoing scrutiny of X by the European Commission under the Digital Services Act (DSA). In December 2025, the Commission issued the first fine under the law against X for breaching transparency obligations. Investigations are continuing to determine whether the platform also violated rules related to the spread of illegal content and online information manipulation.
The Commission has also criticised the platform for enabling the creation of sexualised images and has ordered it to preserve all documents linked to its AI chatbot as part of the ongoing probe.
Deepfake technology relies on advanced artificial intelligence and machine-learning techniques to create highly realistic but fabricated videos, images or audio clips, making it appear as though individuals said or did things they never actually did. While the technology can be used in fields such as entertainment and filmmaking, authorities say misuse is increasing rapidly.
Cybersecurity experts warn that deepfakes are now being used in online scams, propaganda campaigns and identity theft. In some cases, criminals have used AI-generated voices to impersonate company executives and convince employees to transfer funds, underscoring the growing financial and security risks posed by the technology.