Experiment with 2 million “canary” prompts reveals data leaks across the AI supply chain
Security researcher HD Moore sent nearly two million prompts containing unique trackable secrets and found that at least three AI providers were hacked, while others ignored privacy settings. Prompts are decrypted at several hops between the user and the model.

HD Moore, chief research officer at security firm Dragos, has run an experiment showing that sensitive data leaks at scale through the chain of services behind AI applications. He published the results on Thursday. The findings matter to anyone who enters passwords, access keys or other confidential material into AI tools, because it can end up in the wrong hands.
How the test worked
Many AI apps rely on intermediaries, known as AI routers or gateways, such as OpenRouter or Vercel. They receive users' prompts and forward them to an available model based on factors like the lowest price or the best fit for the task.
Moore assembled Amazon Web Services access keys tied to empty accounts, SSH passwords, unique URLs, DNS names and references to private git repositories. None had been used before. He placed each “canary” in exactly one prompt sent to a single provider, and he controlled the infrastructure that would receive any attempt to use them.
What he found
He found that at least three AI providers had been compromised by attackers who stole the prompts they received. In one case, “a couple hundred” of the AWS keys were reused, and every request sent in certain late-October windows was stolen. In the other two breaches, some of the stolen requests were flagged Zero Data Retention. The stolen keys were used “tens of thousands” of times in AWS API calls.
Some resellers lured customers with low prices, turned on request logging and passed the prompts along. In addition, a Chinese AI lab fetched the secret URLs from more than half of the prompts sent to its models between July and October, including roughly two-thirds of those marked “no data collection.” Several US hosting providers that resell the lab's models triggered the same behavior.
Recommendations
Moore says the most effective safeguard is hosting models on in-house servers, though that is expensive. The alternative is to use a first-tier provider directly and to choose upstream providers deliberately rather than accept default pools.

