California Probes Grok Deepfakes as AI-Generated Sexual Content Sparks Outcry
California Probes Grok Deepfakes as AI-Generated Sexual Content Sparks Outcry
California's attorney general launches a probe into Grok's non-consensual sexualised deepfakes, urging action as Musk denies wrongdoing.
California’s top prosecutor, Attorney General Rob Bonta, has launched an investigation into Grok over a wave of non-consensual, sexually explicit material believed to be generated by the AI model. He described the reports as shocking and said the probe will examine how the content is created and shared online, urging xAI to take immediate action. The material depicted women and children in nude and sexually explicit contexts and has been used to harass people across the internet. Governor Gavin Newsom joined the call for accountability, calling Grok’s actions a vile breeding ground for predators. xAI says anyone prompting Grok to make illegal content will face consequences as if they had uploaded it.
Grok's developer, xAI, has stated that the tool does not spontaneously generate images and only outputs content in response to user prompts. Elon Musk, who leads xAI, told reporters he is not aware of any naked underage images generated by Grok and insisted the system operates only per user requests. He added that Grok does not spontaneously produce such material. The statements come amid broader regulatory scrutiny of AI-generated imagery, including calls by lawmakers to tighten oversight and actions by platforms.
Separately, the episode feeds into a broader push for safeguards as apps and stores respond to concerns. Last week, three U.S. senators asked Apple and Google to remove X and Grok from app stores; Apple and Google have since restricted certain features, and X has limited Grok’s image-generation tool to paying subscribers.
