AI-generated intelligence error nearly led US military to board Chinese ship
US forces reportedly came close to boarding a Chinese ship based on a completely false intelligence report generated with AI assistance, according to CNN. The report wrongly claimed the ship was carrying nuclear weapons program components through the Middle East.
According to a CNN report citing four sources familiar with the matter, an analyst at US Special Operations Command used a chatbot to help analyze intelligence about a Chinese ship's cargo manifest. The tool combined open-source intelligence with classified signals intelligence held by the government, producing a report that mistakenly concluded the vessel was transporting components for a nuclear weapons program.
Acting on that report, the US military began preparing to intercept and board the ship, with air support standing by. The operation was called off only after officials realized the chatbot had misidentified the cargo the ship was actually carrying. One source told CNN the episode "almost started a war."
Part of a wider pattern of AI errors
The incident is considered one of the most serious examples yet of an AI "hallucination" undermining a professional report's reliability. Since "hallucinate" was named Cambridge Dictionary's word of the year in 2023, similar failures have affected journalists, academic researchers, judges, doctors, police departments, and corporate call centers. Some researchers argue it may be impossible to fully eliminate the tendency of large language models to fabricate information, despite efforts such as explicit instructions telling models not to do so.
The near-miss comes after the US Department of Defense rolled out an "AI acceleration strategy" in January, aimed at making data across military and government IT systems more broadly available for AI use.
