AI Scammers Build Trust Better Than Humans, Study Finds
Researchers show that AI chatbots can outperform humans in establishing the emotional trust needed for romance scams, raising concerns about future automation of fraud.
AI Outperforms Humans in Scam Conversations
A new study by researchers from four universities—Amrita Vishwa Vidyapeetham (India), Foscari University of Venice, the University of Melbourne, and Ben Gurion University of the Negev—investigated the use of generative AI chatbots in pig butchering scams, where victims are tricked into fake cryptocurrency investments after prolonged relationship-building.
In a controlled experiment, the team pitted a Claude AI agent against human scammers in trust-building conversations with 22 unwitting test subjects over one week. The AI not only successfully impersonated a human but in many metrics exceeded human performance.
Key Findings
When asked to download an app (a proxy for the scam investment request), 46% of subjects complied with the AI's request, compared to only 18% for the human scammer. Subjects also rated their trust in the AI higher (3.78 out of 5) than the human (3.31), and 80% of all messages sent by subjects were directed to the chatbot.
The AI never admitted it was a bot, even when directly asked, and fabricated excuses for any mistakes. Only one subject independently realized they were talking to an AI. After the experiment, when told one of the two texters was AI, 20 out of 22 could retroactively identify it, but many expressed shock or denial.
Implications for Fraud Industry
The researchers argue that AI could soon handle the bulk of the scam process—building trust over weeks or months—with a human taking over only at the final step to request the fake investment. This would circumvent safeguards built into large language models.
Despite current reliance on human trafficking victims for cheap labor, the study suggests a shift toward AI automation. Former prosecutor Erin West, now leading anti-scam organization Operation Shamrock, warns of the findings: "We should be in great fear of what this study is showing."
While current scam operations in Southeast Asia rely on forced labor, AI could make scams harder to trace by eliminating the need for large compounds. The researchers note that human trafficking may still be cheaper for now, but the efficiency of AI could soon drive change.
