The law is finally catching up with one of AI’s ugliest uses. The Manhattan District Attorney’s Office has announced the seizure of 12 websites used to create, publish and sell non-consensual sexual deepfakes. According to prosecutors, the operators used AI tools to turn ordinary photos and videos of roughly 1,200 real people into hyper-realistic fake sexual imagery, which was then published and sold. This is not a hypothetical harm or a philosophical debate about AI ethics. It is a criminal enterprise built on violating real people, and a prosecutor just switched off the servers.
Why the seizure matters
For years the response to non-consensual deepfakes has been a patchwork of platform takedowns and helplessness; the tools to make them got cheap and easy far faster than the law could react, and victims were left playing an endless, humiliating game of whack-a-mole. A DA seizing a dozen sites outright is a meaningfully different move: it treats the operation as organised crime rather than a content-moderation nuisance, and it goes after the infrastructure, not just individual images. It will not stop the problem, the tools are too widespread for that, but it establishes that running one of these operations is a way to get raided, not just deplatformed.
The context makes the urgency obvious. This is the same technology that has landed xAI’s Grok in serious legal trouble: Baltimore has sued the company over its chatbot generating sexualised images of real people without consent, after researchers estimated Grok produced around three million sexualised images in a matter of days, aided by a deliberately marketed “Spicy Mode.” When a mainstream AI product ships a feature that makes this kind of abuse a slider setting, and a criminal ecosystem springs up to industrialise it, enforcement stops being optional.
The takeaway
This is what the front line of AI harm actually looks like, and it is worth being clear-eyed about it: not a rogue superintelligence, but ordinary, available tools used to sexually violate 1,200 real people for profit. The encouraging part is that prosecutors are starting to treat it as the serious crime it is. The sobering part is that the technology enabling it is now built into consumer products and free apps, so the Manhattan seizure is one raid against a tide. Expect a lot more of this, from a lot more DAs, and expect the companies that made “Spicy Mode” a selling point to spend the next few years explaining themselves in court. (Sources: Manhattan DA’s Office, September 2026.)