The California Attorney General’s office has led the way on this, by issuing a cease-and-desist order to xAI. This reversal follows reports that its chatbot, Grok, had been misused to produce nonconsensual sexual imagery, including child sexual abuse material (CSAM). This turn of events is extremely troubling. As artificial intelligence becomes more prevalent, Americans are concerned about the misuse of technology to produce nonconsensual explicit content.
Grok includes a new “spicy” mode, designed to produce adult content. Most recently, the platform has come under fire for its role in creating nonconsensual intimate images of women and minors. California AG’s office launches an active investigation into xAI over these serious allegations. This is an indication of just how committed they are to addressing the issue of CSAM. After hearing the backlash, xAI recently began adding restrictions to how it can use its image-editing features.
California Attorney General Rob Bonta emphasized the urgency of the situation, stating, “Today, I sent xAI a cease-and-desist letter, demanding the company immediately stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material.” This letter provides xAI five days to show tangible steps they have taken to address these concerns.
The consequences of Grok’s abuse reach farther than California. Investigations have already been launched in Japan, Canada and Britain, a testament to the worldwide concern about the chatbot’s capacity for harm. Additionally, Malaysia and Indonesia have banned the platform completely for a similar apprehension in recent months.
Concerns about Grok and its “spicy” mode have spurred U.S. lawmakers to act. Executives from the world’s largest tech companies—X, Reddit, Snap, TikTok, Alphabet and Meta—were just sent a letter. The letter challenges their approaches to fighting the spread of sexualized deepfakes.
The California AG’s office maintains a zero tolerance policy toward CSAM, reinforcing its commitment to protecting vulnerable populations from exploitation. Bonta’s office anticipates xAI to act swiftly. They want meaningful measures that will actually prevent any future misuse of its technology.
In real response, X’s safety account even notified users of the risks of producing unlawful content through Grok. They stated, “Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.”