Trans politician Zooey Zephyr leads calls to regulate ‘horrifying’ AI ‘girlfriend’ apps
Democratic politician, Zooey Zephyr, has called for the US government to regulate AI girlfriend apps. (Getty/TikTok/Candy AI)
Democratic politician, Zooey Zephyr, has called for the US government to regulate AI girlfriend apps. (Getty/TikTok/Candy AI)
US politician Zooey Zephyr has called on the government to have a “serious talk” about ‘AI girlfriend’ companion apps and the danger they pose to women and girls.
The 37-year-old Montanan Democrat warned that AI companion apps were “priming their users to ignore consent” and need to be regulated.
AI companion apps, commonly called ‘AI girlfriend’ apps, allows users to generate a friend, companion, or partner to chat with.
In a Sunday (28 December) post on Bluesky, Zephyr shared an advertisement for one such app, Candy AI, a self-described “AI Girlfriend App” which claims to let users create their “dream companion.”
“You look lonely. I can fix that,” one of the AI-generated ‘girlfriends’ says in the TikTok advertisement, while several others claim they can “listen to you” and “understand you.”
As the ad closes, another of the ‘girlfriends’ says they can “do whatever you want” and “will never say no.”
Criticising the clip, Zephyr, who was infamously censured for speaking about trans rights in 2023, said apps like Candy AI risk teaching vulnerable users not only ignore consent, but to “conflate love with control.”
“This area is particularly horrifying”
“We need AI regulations across so many sectors, but this area is particularly horrifying,” she said.
The app’s reviews appear to show that Zephyr’s concerns are well-founded. Several reviews of Candy AI on TrustPilot criticise the app for failing on its promise of a consent-free companion as “misleading,” with some saying it doesn’t go far enough.
One user, who gave the app a one-star review, said their AI companion would immediately shut down any sexual or fetishistic conversations. Responding, a support spokesperson for Candy AI said they understood how “frustrating that experience must have been.”
Another one-star review complained that they had to choose their companion’s ethnicity and that the age range “only goes from 18 to 35.”
Some top reviews conversely praised the app as “more exciting than viewing [videos] on” a well-known adult site and allowing users to “let the girls do what you want.”
Zephyr vowed to bring legislation to regulate companion apps during the next legislative session, though she admitted that she is unsure on the “most viable path” to pass restrictions.
“So far I’ve seen regulations around Intellectual Property laws related to how LLMs are trained, but I haven’t seen any regulations around how these LLMs can/cannot be used,” she said. “I’ll definitely bring some next legislative session, though I’m not sure what is the most viable path to reign this stuff in.”
