Finance ministers, central bankers and financiers have expressed serious concerns about a powerful new AI model they fear could undermine the security of financial systems.
The development of the Claude Mythos model by Anthropic has led to crisis meetings, after it found vulnerabilities in many major operating systems.
Experts say it potentially has an unprecedented ability to identify and exploit cyber-security weaknesses - though others caution further testing is needed to properly understand its capabilities.
Canadian Finance Minister François-Philippe Champagne told the BBC that Mythos had been discussed extensively at the International Monetary Fund (IMF) meeting in Washington DC this week.
“Certainly it is serious enough to warrant the attention of all the finance ministers,” he said.
I’m not sure I get the concern. If there are vulnerabilities they have probably been sold to NSA, other state hackers and black hats already. Mythos would help close them for everyone.
Sure, a bad actor could use it to break in, but Mythos is not some secret hacking tool, it’s an expensive LLM you can run against your own code and system giving you the upper hand.
Anthropic is actually acting responsibly by contacting maintainers and platforms with bugs and the possibility to analyze their systems before it’s released to the wider public. And if it’s all hype then this is a money grabbing operation to finally make good money off of LLMs. That concern however doesn’t seem to be shared by the financiers.
I bought a game server once that was hosted on a VPS and I didn’t bother doing much security setup for the first 12 hours (fail2ban etc). I was SSH-ing into it so the address was kind of ‘open’, but not listed anywhere.
Got over 300 failed SSH attempts in 12 hrs before I set up fail2ban and set up keys. Just massive botnets scouring addresses. (The game server never took off so it’s actually now being used as a honeypot named like a payment node so I can report a bunch).
The worry is more the automation. Someone could cast an extremely wide net and find vulnerabilities that people didn’t even know could be vulnerabilities. If you could run 1000 agents each one with the ability of a network/SE expert, you can steal a lot of things very quickly.
I’m worried about the tons of barely maintained software run by your average company. Most commercial software is made by relatively small outfits and is drowning technical debt. The only thing saving their customers is the effort of picking through it.
But now any loser with a decompiler and a $100 Claude sub can ruin a whole lot of people’s day.
Things will get better, but the near term is pretty fucked.
I don’t have it handy, but I recommend reading Anthropic’s report about mythos and security. They state that in the long run, models which can iteratively build an attack against a perceived vulnerability will be a major win for defenders, but in the short term, they present an advantage to attackers since they basically expose oodles of new zero days.
Maybe the concern is less scrupulous actors will soon be able to develop similar models?
Yep; same as it ever was. The arms race continues. At least until Butlerian Jihad arrives.