Anthropic CEO Dario Amodei denies calls to ban open-weight models, but fears Chinese AI

Anthropic founder and CEO Dario Amodei responded on Monday to industry rumors that his company supports U.S. government efforts to ban Chinese open-weight models or open-weight models in general. In a blog post, he emphasized: "Anthropic has never advocated for a ban on open-weights models." This came after Nvidia CEO Jensen Huang posted an open letter on Friday, signed by Nvidia and other AI companies including Hugging Face, Meta, Microsoft, and Mistral, urging policymakers to avoid broad "premature restrictions" on open-weight AI models. While the letter did not specifically mention China, the industry debate centers on allegations that Chinese AI labs are advancing, often by stealing intellectual property from U.S. counterparts, including through distillation.

Amodei stated that open-weight models without dangerous capabilities are a public good: they cost nothing beyond compute resources and provide value to businesses, developers, and researchers. However, he has long-standing fears about AI, particularly from authoritarian governments. "My fear is that authoritarian governments will build models more powerful than those in the U.S. to achieve permanent military superiority and/or repress their own people," he wrote. He noted that the Chinese Communist Party (CCP) is not the only authoritarian government he worries about, but it is the "most capable." Amodei also fears AI could enable biological attacks, not just cybersecurity threats, and believes open-weight models are more dangerous in those scenarios because it is difficult to apply guardrails or monitor usage.

Citing a UK AI Security Institute report, he pointed out that once open-weight models are released, they cannot be withdrawn, contradicting open-source advocates' arguments that access to powerful models helps defenders. Amodei listed actions to thwart China, including restricting its access to powerful chips (a longstanding U.S. policy) and cracking down on distillation. The U.S. has threatened sanctions if IP theft involving U.S. models is found. He also supported efforts to create a model safety testing organization, especially if it includes global participation, including China. "To be effective, testing would need to be global, which means even the CCP would need to be on board. I think this may actually be possible… limited cooperation around preventing AI biological weapons may be possible because it is in China’s interest too," Amodei wrote.