The Union Ministry of Electronics and Information Technology issued a formal notice to X after its AI platform Grok was misused to create "obscene" images.
In the notice, the ministry asked the Elon Musk-owned social media platform to remove "obscene, nude, indecent and sexually explicit content" in 72 hours and carry out an immediate comprehensive review of Grok's 's "technical, procedural and governance-level" frameworks.
"It has especially been observed that the service namely 'Grok AI' developed by you and integrated and made available on the X platform is being misused by users to create fake accounts to host, generate, publish or share obscene images or videos of women in a derogatory or vulgar manner in order to indecently denigrate them," read the letter from the ministry.
It said that such violations are failure of safeguards and enforcement mechanisms, pointing out that it amounts to gross misuse of AI. It also added that there are serious lapses in X statutory due diligence obligations under the Information Technology Act, 2000 and the IT (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021.
The central government also asked X to submit a detailed action taken report to the ministry. The ministry further warned that failure to comply could result in X losing its safe harbour protection under Section 79 of the IT Act.
This comes after Union minister Ashwini Vaishnaw said on Friday that social media firms should take responsibility for content they publish and a standing committee has already recommended a tough law to fix accountability of platforms.
"Social media should be responsible for the content they publish. Intervention is required," Vaishnaw said while replying to a question on Grok misuse.
Rajya Sabha member Priyanka Chaturvedi has also written to Vaishnaw asking for urgent intervention on increasing incidents of AI apps being misused to create vulgar photos of women.
