Finance ministers, central bankers and financiers have expressed serious concerns about a powerful new AI model they fear could undermine the security of financial systems.
The development of the Claude Mythos model by Anthropic has led to crisis meetings, after it found vulnerabilities in many major operating systems.
Experts say it potentially has an unprecedented ability to identify and exploit cyber-security weaknesses - though others caution further testing is needed to properly understand its capabilities.
Canadian Finance Minister François-Philippe Champagne told the BBC that Mythos had been discussed extensively at the International Monetary Fund (IMF) meeting in Washington DC this week.
“Certainly it is serious enough to warrant the attention of all the finance ministers,” he said.
Are you certain about that? Anthropic has a team of security engineers “validating” the LLM output, and then they have been passing on their “validated” outputs to third-party security researchers to “confirm” them.
Tellingly, they don’t say how many false positives have to be filtered through in order to find the correct vulnerabilities with working exploits, but I imagine that if all those security researchers were tasked with auditing the same codebases, they would probably find the same (or more) vulnerabilities without the shotgun guessing of an LLM to guide them.
You need to remember that these claims are being made by a company that has enormous financial incentive to make everyone believe that this model is a huge breakthrough.
I have no inside knowledge on this particular work, but their previous work on the OSS-fuzz targets and on Firefox were all excellent quality bug reports.
Seriously. Look them up.
They were all reproducible ways to trigger faults in ASan builds. That’s by definition memory corruption. We can argue about whether all of them are exploitable, but a) they need to get fixed regardless b) we know that even tiny memory corruptions can often be leveraged into a compromise given enough effort.
Yes of course they were. Professional security researchers tend to produce professional, high quality reports.