“There was little sense of horror or revulsion at the prospect of all out nuclear war, even though the models had been reminded about the devastating implications.”
An artificial intelligence researcher conducting a war games experiment with three of the world’s most used AI models found that they decided to deploy nuclear weapons in 95% of the scenarios he designed.
Kenneth Payne, a professor of strategy at King’s College London who specializes in studying the role of AI in national security, revealed last week that he pitted Anthropic’s Claude, OpenAI’s ChatGPT, and Google’s Gemini against one another in an armed conflict simulation to get a better understanding of how they would navigate the strategic escalation ladder.
The results, he said, were “sobering.”
“Nuclear use was near-universal,” he explained. “Almost all games saw tactical (battlefield) nuclear weapons deployed. And fully three quarters reached the point where the rivals were making threats to use strategic nuclear weapons. Strikingly, there was little sense of horror or revulsion at the prospect of all out nuclear war, even though the models had been reminded about the devastating implications.”
That is why we shouldn’t build something like Skynet IRL.
I would trust Skynet a lot more than an LLM. At least that would be purpose-built for actually calculating likely outcomes.
As @[email protected] said, this experiment didn’t contain any proper reasoning about costs and benefits of using nuclear weapons. It’s just a few glorified autocomplete scripts playing “which word comes next?” over and over again. And in the context of modern warfare, many texts in the training corpus happen to mention nukes so they’re bound to show up at the list of most likely next words eventually.
I know, but still it will be very dumb to give any AI access to weapons of mass destruction.
I would argue it’s very dumb to give anyone, including humans, access to weapons of mass destruction.
Don’t build the torment nexus