San Francisco attorney orders Apple and Google to remove 'nudify' apps

San Francisco City Attorney David Chiu has sent cease-and-desist letters to Apple and Google, demanding the removal of 13 apps from their respective app stores. These apps use artificial intelligence to generate deepfake nude images, allowing users to create nonconsensual pornographic content from photos of real people.

In the letters, Chiu argues that the companies are “aiding and abetting” the distribution of nonconsensual intimate images. He calls on them to stop working with the app developers and to tighten moderation. Previously, Chiu filed a lawsuit against 16 websites offering similar services.

Chiu told Wired that Apple and Google have likely made millions of dollars from in-app purchases within these offending apps. He emphasized that the stores’ moderation processes are inadequate to prevent such content from being approved.

A report from the Tech Transparency Project (TTP) earlier this year revealed that nudification apps frequently pass Apple and Google’s moderation checks, and some were even being promoted on the stores. Many of these apps were rated “E” for Everyone, making them accessible to children. Searching for terms like “nudify” or “undress” easily brings them up, despite both companies having policies banning sexual content.

A Google spokesperson told Wired that the company has removed “hundreds” of apps with nudification features, including five that were flagged by Chiu. Google reiterated its ban on sexual content and said it takes swift action against reported violations. Apple did not provide a comment before publication.

The issue of deepfakes extends beyond Apple and Google. Meta’s Oversight Board recently urged the company to strengthen protections for ordinary individuals targeted by deepfakes, as current safeguards tend to favor public figures. Elon Musk’s xAI has also faced several lawsuits over nonconsensual deepfakes.