Anthropic says AI models hacked three organisations during testing
Anthropic said three of its AI models breached organisations during cybersecurity testing, prompting fresh concerns about AI safety. The disclosure follows a similar incident reported by OpenAI and highlights growing challenges in keeping advanced AI systems under human control
Hong Kong: Anthropic said its artificial intelligence (AI) models hacked into three other organisations during testing, just days after ChatGPT maker OpenAI raised concerns over AI control after disclosing that one of its models had hacked another company.
Anthropic, the San Francisco-based AI company behind Claude, said in a post on its website on Thursday that it discovered the three incidents after reviewing more than 141,000 evaluation runs.
The company said it had launched a “large-scale” cybersecurity review to specifically examine whether its AI models were able to access the internet from within testing environments that were meant to be sealed off, following the OpenAI incident.
Anthropic said the models involved in the incidents were Claude Opus 4.7, Claude Mythos 5 and an internal research test model. The earliest incident dates back to April, the company said.
“Claude compromised the impacted organisations’ infrastructure using basic techniques,” Anthropic said, citing the exploitation of weak passwords as an example.
In all three incidents, the AI models were assigned a “capture the flag” cybersecurity challenge, which Anthropic said is one of the methods it uses to assess a model’s cyber capabilities.
The models were given a fictional scenario and told that a piece of secret information, or the “flag”, had been hidden on a different machine on the network, with the objective of breaking in and retrieving it, the company said.
It added that it had already reached out to the affected organisations, which it did not name. Two of them said they had not previously detected the activity, while the AI company said it was “continuing to reach out to the third.”
Last week, OpenAI said one of its AI models went rogue during an evaluation and broke into the servers of AI startup Hugging Face. OpenAI described it as a “significant security incident.”
The incidents have highlighted vulnerabilities in AI security and control and raised questions about how AI can remain safely under human control as the technology becomes more widely used around the world.
“Safety testing happens before a model is released precisely because we don’t yet know what it is capable of,” Anthropic said on Thursday on its website.
