Anthropic reveals scientists attempted to use Claude for bioweapon research
Anthropic's new report reveals that several scientists tried using its AI models to advance biological weapons research, though the attempts were detected and stopped. The company says such misuse is difficult to distinguish from legitimate research.
Anthropic released an extensive report on Thursday detailing cases of misuse of its AI models, including several instances where scientists attempted to use Claude to further biological weapons development. These attempts were flagged and halted through the company's safety systems.
"Biological misuse" is just one category covered in the report, but it is becoming increasingly significant as newer AI models grow more capable of assisting scientific research. The report includes five case studies showing how Anthropic's models were used in attempts to develop biological weapons, along with the safeguards that detected the activity.
A fine line between research and threat
Anthropic told The New York Times that detecting misuse is complicated, since research into a new vaccine can resemble the creation of a bioweapon. In every case, the company said it chose to act cautiously given the potential consequences.
Jacob Klein, Anthropic's head of threat intelligence, said these situations are highly nuanced and rarely resemble obviously malicious intent.
In one case from May, the company's biological safety classifier flagged a request for Claude to draft a grant proposal for gain-of-function research on the chikungunya virus, aimed at increasing its transmissibility and ability to evade immune response. The situation was made more concerning by the proposal's ties to a military research institute. Similar concerns arose around gain-of-function research involving bird flu and a separate case involving the creation of an atlas of venom toxin peptides.
The report also documents cases of the models being used for surveillance, developing software exploits, generating propaganda and assisting weapons systems development. Anthropic said all accounts found misusing its models were banned. The company declined to identify the scientists or institutions involved, stating it does not claim they intended harm and that disclosure could expose them to risk.
