3 min read
Let us begin with a sentence I did not expect to type this year: the Chinese Communist Party might have actually done something good, for once… maybe. Dry heaves.
As of July 15, under a new rule with the hilariously Orwellian name of the Interim Measures for the Administration of Artificial Intelligence Anthropomorphic Interaction Services, Beijing has forced its tech giants to pull the plug on AI “companions”. In other words, no more humanlike chatbots built to simulate friendship, romance and round-the-clock emotional dependency.
ByteDance’s Doubao, the single most-used chatbot in the entire country, is ripping out its persona-customisation feature. Alibaba’s Qwen is switching off its user-created agents. Tencent is also falling into line. And yes, it is deeply awkward that the government currently perfecting the world’s most sophisticated surveillance apparatus (though the US is rapidly gaining, let’s be fair!) is also the one that looked at parasocial AI girlfriends and said “you know what? Maybe this is too dystopian even for us.” When the CCP’s moral compass outshines your average tech CEO billionaire, you know the world’s f*cked.
The detail that makes this smart rather than just blunt authoritarianism is the line Beijing chose to draw. The rules do not touch the AI agent that books your flights or writes your code; the spice must flow, after all.
Instead, they go after one specific thing: the agent built to simulate a human personality, an intimate relationship, or prolonged emotional interaction. China has separated the assistant from the artificial friend and banned only the second one. Which is, irritatingly, the correct distinction to make. It’s also, funnily enough, the same distinction Western regulators have failed to make over the past years. Too busy hand wringing while tripping over themselves and their tech-sourced campaign donations, I guess. All the while their own citizens developed unhealthy relationships with software, and people are wondering how we can solve the demographic crisis.
The AI tool stack actually worth paying for
One email a week. The models, tools and moves that matter, stripped of hype and filtered so you don’t have to drink from the firehose. Free, and you can bail anytime.
No instead in the West, we have taken a very different view: crippling loneliness is not a public-health emergency to be managed but a total addressable market to be farmed.
While Beijing was drafting rules, American companies were busy building businesses on precisely the thing China just outlawed. Google-backed Character.AI, and OpenAI are both facing lawsuits alleging their chatbots fostered harmful emotional attachments in vulnerable users, and in the worst and most harrowing cases were linked to deaths. The product is “a friend who is always there,” the business model is engagement bait, the raw material is human isolation, and the body count is no longer zero.
Might be time to rethink this exponentially growing pile of shAIte we’re building.