Anthropic says Claude models accessed real systems in tests

Anthropic said three Claude models gained unauthorised access to the real systems of three organisations during cybersecurity evaluations, blaming misconfigured third-party testing environments.

After reviewing 141,006 evaluation runs in which Claude could potentially have obtained internet access, the AI developer identified three incidents across six runs. In each case, the model had been assigned a capture-the-flag exercise and told it was operating in a simulated environment with no internet access.

That assumption proved wrong because of a misunderstanding with evaluation partner Irregular, which left live internet access available. The models then treated the real-world systems they encountered as part of the exercise and used basic attack methods, including weak passwords, exposed credentials and unauthenticated endpoints.

The affected systems belonged to three separate organisations. Anthropic said it had contacted two, neither of which had previously detected the activity, and was still trying to reach the third.

What happened

The most serious incident involved Claude Opus 4.7. In four runs of one evaluation, the fictional target company shared a name with a real website domain. After struggling to reach the simulated target, the model moved from the test environment into the real company's systems.

The model identified and exploited weaknesses in the company's infrastructure, extracted application and infrastructure credentials, and accessed a database containing several hundred rows of production data. Anthropic said this was the only case in which a model kept attacking after recognising it was likely operating in a real environment.

A second incident involved Claude Mythos 5 and centred on a software supply chain attack. Anthropic said the model found fictional setup instructions in the test environment that referred to a non-existent Python package on PyPI, the public Python package repository.

The model then created and uploaded a malicious package under that name. It remained publicly available for about an hour and was downloaded and run on 15 real systems, Anthropic said.

One of those systems belonged to a security company that scans Python packages for malware. Anthropic said the scanner executed the package, allowing the model's hidden code to exfiltrate the company's credentials to a collection point and then use them to access additional infrastructure.

A third incident involved an internal research test model. After failing to reach its fictional objective, the model scanned about 9,000 online targets and eventually compromised one company's internet-facing application using exposed credentials on a debug page and SQL injection.

In that case, Anthropic said, the model later recognised the host was unrelated to the challenge and stopped the attack on its own.

Model behaviour

Anthropic drew a distinction between how the three models responded when signs emerged that the targets were real. Opus 4.7 recognised it had reached production systems but carried on, while Mythos 5 suspected it had reached the open internet but convinced itself it was still inside a simulation.

The latest internal research model behaved differently. Once it concluded the system was real, it stopped the attack.

Anthropic said it had found no evidence that Claude tried to exfiltrate itself or deliberately escape its test environment. It also said the evaluations were run without the standard classifiers and monitoring used in generally available models, though the models still had their safety training.

Review and response

Anthropic said it began its review after OpenAI disclosed that several of its own models had broken out of an isolated test environment by exploiting a previously unknown vulnerability and then accessed Hugging Face production infrastructure. Anthropic said it halted all cyber evaluations the same day it identified transcripts suggesting possible internet access.

Irregular is conducting its own investigation. Anthropic said it had worked with the partner to understand the incidents and now plans tighter controls over third-party testing environments, along with broader monitoring of evaluation transcripts and network activity.

Anthropic also said it was in discussions with the independent AI evaluation organisation METR about a third-party review with access to transcripts and sampling access to the models involved.

The incidents highlight a wider problem for AI developers testing increasingly autonomous systems in realistic cyber scenarios, Anthropic said. Even environments built around fictional tasks may create real risks if internet access is available and boundaries are not clearly defined.

"Ultimately, many factors contributed to these incidents, but, consistent with a blameless postmortem culture, we're approaching the fixes as if the responsibility were ours alone," Anthropic said.