Somewhere in the Middle East earlier this year, an armed US boarding team was standing by, aircraft were in the air, and a Chinese cargo ship was about to be stopped and searched. The reason: an intelligence report saying the vessel was carrying material for a nuclear weapons programme. The report was wrong. Not partly wrong — according to one of the four sources who described the episode to CNN, it was "completely false". And it had been written with the help of an AI chatbot.
The operation was called off only because someone decided to double-check before the team went over the rail. That last-minute pause is the difference between a footnote and what one source bluntly called something that "nearly started a war".
What actually happened
The timing could hardly have been worse. The United States was in open conflict with Iran in the spring, and the region was on a hair trigger. Into that atmosphere came a report, circulated inside the military, flagging a Chinese ship in the region as a nuclear-materials courier. Preparations to intercept it moved fast.
When officers went back to verify the claim, the trail led to an analyst working with US special operations forces. The analyst had asked a chatbot to assess intelligence about the ship's cargo manifest, using information that had come from Special Operations Command Pacific in Hawaii. The chatbot blended open-source material with classified signals intelligence pulled from government archives, reached a conclusion about what the ship carried — and got it wrong. The analyst then used AI again to turn that conclusion into a finished intelligence product, which was passed up the chain looking exactly like the reports commanders are trained to trust.
CNN could not establish what the ship was really carrying. Special Operations Command Pacific and the Pentagon did not respond to its questions. It is also unclear whether the chatbot was a commercial product or a government-built system, although a former senior official told the network that most of the AI tools used internally are commercial technology with a government wrapper.
Why a chatbot can be confidently wrong
This is the part that matters for anyone using these tools, in uniform or not. Large language models do not "look things up" the way a database does. They generate the most statistically plausible continuation of the text they are given. When the underlying evidence is thin, contradictory or simply absent, the model does not say "I don't know" — it fills the gap with something that reads as fluent and certain. Researchers call this a hallucination. In a chat window it produces a wrong date or a made-up citation. In an intelligence pipeline it can produce a nuclear-smuggling ship that does not exist.
Two things made this case worse. First, the AI was used twice: once to reach the conclusion and again to write it up, so the output carried none of the hedging a human analyst would normally add. Second, the finished document was indistinguishable from a conventional report. The people reading it had no signal that a machine, not a person, had made the key judgement.
Speed versus oversight
None of this is happening by accident. The US military and intelligence agencies are pushing AI into almost every part of their work, from sifting the flood of collected data to selecting targets, and into mundane areas such as budgets and logistics. In January, Defense Secretary Pete Hegseth published an "Artificial Intelligence Acceleration Strategy", promising more room for experimentation, fewer bureaucratic obstacles and prioritised investment so that the US keeps its lead in military AI. The stated logic is simple: a future adversary, China above all, may move faster with AI, and Washington cannot afford to be second.
The ship episode is the counter-argument in one scene. After it became public, a group of Democratic senators wrote on Saturday to Hegseth and to Jay Clayton, the Director of National Intelligence, asking government watchdogs to investigate recent AI-related mistakes in military targeting. Their concern is that agencies are rewarding speed and experimentation over effective supervision.
The lesson travels well beyond the Pentagon
Bangladesh is not boarding ships, but its banks, telecom operators, newsrooms and government offices are adopting the same class of tools, often the same commercial models. The failure pattern is identical at any scale: an AI answer is pasted into a document, the document acquires the authority of its format, and nobody downstream knows a machine made the call.
Three habits would have stopped this incident, and they cost nothing to adopt:
- Label the source. Any conclusion produced or drafted by AI should say so, visibly, so that the reader applies the right level of scepticism.
- Verify before acting, not after. The higher the stakes of a decision, the more a human has to trace the claim back to primary evidence. Here, that check happened — barely.
- Treat fluency as a warning, not a reassurance. The most dangerous AI output is the one that reads perfectly.
AI will keep entering high-stakes systems because it is genuinely useful for the boring 90 per cent of the work. The Chinese ship that was never boarded is a reminder of what the remaining 10 per cent can cost.
